Multi-Objective DRL for Efficient Kubernetes Microservice Scheduling
DOI:
https://doi.org/10.15837/ijccc.2026.5.7222Keywords:
microservice, multi-objective microservice deployment model, deep reinforcement learning, elastic scalingAbstract
Container based microservice architecture is becoming the priority of service development in edge computing. Its flexibility and scalability could provide stable service response. However, due to the limited computing resources of each edge node in edge computing, how to use computing resources efficiently has become an important issue in microservice deployment. This study formulates the multi-objective microservice deployment problem (MMDP) in Kubernetes, balancing service latency and resource utilization. We propose a deep reinforcement learning method with reward shaping and heuristic scaling to derive optimal deployment policies. Experiments show that our approach reduces response time by 25–30% and improves load balancing by 20% compared with Kubernetes default scheduling and DQL. These results demonstrate the applicability of reinforcement learning for distributed computing and automatic control in microservice systems.
References
Altaf, U.; Jayaputera, G.; Li, J.; et al. (2018). Autoscaling a defence application across the cloud using docker and kubernetes, IEEE/ACM International Conference on Utility and Cloud Computing Companion, Zurich, Switzerland, pp. 327-334, 2018. https://doi.org/10.1109/UCC-Companion.2018.00076
Brogi, A.; Forti, S.; Guerrero, C.; Lera, I. (2020). How to place your apps in the fog: State of the art and open challenges, Software: Practice Experience, 50(5), pp. 719-740, 2020. https://doi.org/10.1002/spe.2766
Burer, S.; Letchford, A.N. (2012). Non-convex mixed-integer nonlinear programming: A survey, Surveys in Operations Research and Management science, 17(2), pp. 97-106, 2012. https://doi.org/10.1016/j.sorms.2012.08.001
Burns, B.; Grant, B.; Oppenheimer, D.; Brewer, E.; Wilkes, J. (2016). Borg, omega, and kubernetes, Queue, 14, pp. 70-93, 2016. https://doi.org/10.1145/2898442.2898444
Cai, Z.C.; Buyya, R. (2022). Inverse queuing model based feedback control for elastic container provisioning of web systems in Kubernetes, IEEE Transactions on Computers, 71(2), pp. 337-348, 2022. https://doi.org/10.1109/TC.2021.3049598
Chen, S.; Yuan, Q.; Li, J.; He, H.; Li, S.; Jiang, X.; Yang, J. (2024). Graph neural network aided deep reinforcement learning for microservice deployment in cooperative edge computing, IEEE Transactions on Service and Computing, 17(6), pp. 3742-3757, 2024. https://doi.org/10.1109/TSC.2024.3417241
Chen, X.; Bi, Y.; Chen, X.; Zhao, H.; Cheng, N.; Li, F.; Cheng, W. (2022). Dynamic service migration and request routing for microservice in Multicell mobile-edge computing, IEEE Internet of Things Journal, 9(15), pp. 13126-13143, 2022. https://doi.org/10.1109/JIOT.2022.3140183
Deng, L.; Wang, Z. Y.; Sun, H.Y.; Li, B.; Yang, X. (2023). A deep reinforcement learning-based optimization method for long-running applications container deployment, International Journal of Computers Communications & Control, 18(4), pp. 5013, 2023. https://doi.org/10.15837/ijccc.2023.4.5013
Deng, S.; Xiang, Z.; Taheri, J.; Khoshkholghi, M.A.; Yin, J.; Zomaya, A.Y.; Dustdar, S. (2021). Optimal application deployment in resource constrained distributed edges, IEEE Transactions on Mobile Computing, 20(5), pp. 1907-1923, 2021. https://doi.org/10.1109/TMC.2020.2970698
Du, T.; Zhang, Y.; Shi, X.; Chen, S. (2018). Multiple slice k-space deep learning for magnetic resonance imaging reconstruction, 42nd Annual International Conference IEEE Engineering in Medicine and Biology Society, Montreal, QC, Canada, pp. 1564-1567, 2020. https://doi.org/10.1109/EMBC44109.2020.9175642
Duan, H.Y.; Wu, Z.; Ji, S.; Chen, Z.; Jiang, C.X.; Zhang, W. (2025). Distributed microservice deployment for satellite edge computing networks: A multi-agent deep reinforcement learning approach, IEEE Transactions on Vehicular Technology, 2025. https://doi.org/10.1109/TVT.2025.3565270
Gao, W.; Ye, Z.; Sun, P.; Zhang, T.; Wen, Y. (2024). UNISCHED: A unified scheduler for deep learning training jobs with different user demands, IEEE Transactions on Computers, 73(6), pp. 1500-1515, 2024. https://doi.org/10.1109/TC.2024.3371794
Guo, F.; Tang, B.; Tang, M. (2022). Joint optimization of delay and cost for microservice composition in mobile edge computing, World Wide Web: Internet and Web Information Systems, 25(5), pp. 2019-2047, 2022. https://doi.org/10.1007/s11280-022-01017-2
Hoang, M.T.; Nguyen, N.V.; Pham, T.A.; Nguyen, T.T.; Dang, T.M.; Nguyen, H.N. (2024). Evaluating Dimensionality Reduction Methods for the Detection of Industrial IoT Attacks in Edge Computing, International Journal of Computers Communications & Control, 19(5), pp. 6767, 2024. https://doi.org/10.15837/ijccc.2024.5.6767
Hu, M.; Wang, H.; Xu, X.; He, J.; Hu, Y.; Deng, T.; Peng, K. (2024). Joint optimization of microservice deployment and routing in edge via multi-objective deep reinforcement learning, IEEE Transactions on Network and Service Management, 21(6), pp. 6364-6381, 2024. https://doi.org/10.1109/TNSM.2024.3443872
Huang, J.; Xiao, C.; Wu, W. (2020). RLSK: A job scheduler for federated Kubernetes clusters based on reinforcement learning, 2020 IEEE International Conference on Cloud Engineering, Sydney, NSW, Australia, pp. 116-123, 2020. https://doi.org/10.1109/IC2E48712.2020.00019
Ionescu, S.A.; Diaconita, V. (2023). Transforming Financial Decision-Making: The Interplay of AI, Cloud Computing and Advanced Data Management Technologies, International Journal of Computers Communications & Control, 18(6), pp. 5735, 2023. https://doi.org/10.15837/ijccc.2023.6.5735
Jiang, Y.; Wang, Z.; Jin Z.(2023). Iot Data Processing and Scheduling Based on Deep Reinforcement Learning, International Journal of Computers Communications & Control, 18(6), pp. 5998, 2023. https://doi.org/10.15837/ijccc.2023.6.5998
Jiao, Z.; Zhang, J.; Yao, P.; Wan, L.; Ni L. (2020). Service deployment of C4ISR based on genetic simulated annealing algorithm, IEEE Access, 8, pp. 65498-65512, 2020. https://doi.org/10.1109/ACCESS.2020.2981624
Li, B.; He, Q.; Cui, G.; Xia, X.; Chen, F.; Jin, H.; Yang, Y. (2022). READ: Robustness-oriented edge application deployment in edge computing environment, IEEE Transactions on Services Computing, 15(3), pp. 1746-1759, 2022. https://doi.org/10.1109/TSC.2020.3015316
Lv, W.; Wang, Q.; Yang, P.; Ding, Y.; Yi, B.; Wang, Z.; Lin, C. (2022). Microservice deployment in edge computing based on deep Q learning, IEEE Transactions on Parallel and Distributed Systems, 33(11), pp. 2968-2978, 2022.
Ma, X.; Yao, T.; Hu, M.; Dong, Y.; Liu, W.; Wang, F.; Liu, J. (2019). A survey on deep learning empowered IoT applications, IEEE Access, 7, pp. 181721-181732, 2019. https://doi.org/10.1109/ACCESS.2019.2958962
Ma, X.; Zhou, A.; Zhang, S.; Wang, S. (2020). Cooperative service caching and workload scheduling in mobile edge computing, IEEE International Conference on Computer Communications, Toronto, ON, Canada, pp. 2076-2085, 2020. https://doi.org/10.1109/INFOCOM41043.2020.9155455
Maia, A.M.; Ghamri-Doudane, Y.; Vieira, D.; de Castro, M.F. (2019). Optimized placement of scalable IoT services in edge computing, Proceeding IFIP/IEEE Symposium on Integrated Network and Service Management, Arlington, VA, USA, pp. 189-197, 2019.
Mao, Y.; Fu, Y.; Zheng, W.; Cheng, L.; Liu, Q.; Tao, D. (2022). Speculative container scheduling for deep learning applications in a Kubernetes cluster, IEEE Systems Journal, 16(3), pp. 3770- 3781, 2022. https://doi.org/10.1109/JSYST.2021.3129974
Mart, O.; Negru, C.; Pop, F.; Castiglione, A. (2020). Observability in Kubernetes cluster: Automatic anomalies detection using Prometheus, 22nd IEEE International Conference on High Performance Computing and Communications, Yanuca Island, Cuvu, Fiji, pp. 565-570, 2020. https://doi.org/10.1109/HPCC-SmartCity-DSS50907.2020.00071
Nadareishvili, I.; Mitra, R.; McLarty, M.; Amundsen M. (2016). Micro Service Architecture: Aligning Principles, Practices, and Culture, Newton, MA, USA: O'Reilly Media, 2016.
Pallikonda, A.K.; Bandarapalli, V.K.; Aruna, V. (2025). Enhancing performance and reducing latency in autonomous systems through edge computing for real-time data processing, Mechatronics and Intelligent Transportation Systems, 4(3), pp. 154-165, 2025. https://doi.org/10.56578/mits040305
Peng, K.; Wang, L.; He, J.; Cai, C.; Hu, M. (2024). Joint optimization of service deployment and request routing for microservices in mobile edge computing, IEEE Transactions on Services Computing, 17(3), pp. 1016-1028, 2024. https://doi.org/10.1109/TSC.2024.3349408
Peng, Y.; Bao, Y.; Chen, Y.; Wu, C.; Meng, C.; Lin, W. (2021). DL2: A deep learning-driven scheduler for deep learning clusters, IEEE Transactions on Parallel and Distributed Systems, 32(8), pp. 1947-1960, 2021. https://doi.org/10.1109/TPDS.2021.3052895
Poularakis, K.; Llorca, J.; Tulino, A. M.; Taylor, I.; Tassiulas, L. (2020). Service placement and request routing in MEC networks with storage, computation, and communication constraints, IEEE/ACM Transactions on Networking, 28(3), pp. 1047-1060, 2020. https://doi.org/10.1109/TNET.2020.2980175
Quang, P.T.A.; Hadjadj-Aoul, Y.; Outtagarts, A. (2019). A deep reinforcement learning approach for VNF forwarding graph embedding, IEEE Transactions on Network and Service Management, 14(4), pp. 1318-1331, 2019. https://doi.org/10.1109/TNSM.2019.2947905
Samanta, A.; Tang, J. (2020). Dyme: Dynamic microservice scheduling in edge computing enabled IoT, IEEE Internet of Things Journal, 7(7), pp. 6164-6174, 2020. https://doi.org/10.1109/JIOT.2020.2981958
Saha, S.; Perumal, I.; Abbas, M.; Manimozhi, I.; Bhat, C.R. (2024). Contextual information based scheduling for service migration in mobile edge computing, International Journal of Computers Communications & Control, 19(3), pp. 6143, 2024. https://doi.org/10.15837/ijccc.2024.3.6143
Schneider, S.; Khalili, R.; Manzoor, A.; Qarawlus, H.; Schellenberg, R.; Karl, H.; Hecker, A.
(2020). Self-learning multi-objective service coordination using deep reinforcement learning, IEEE Transactions on Network and Service Management, 18(3), pp. 3829-3842, 2020. https://doi.org/10.1109/TNSM.2021.3076503
Schneider, S.; Qarawlus, H.; Karl, H. (2021). Distributed online service coordination using deep reinforcement learning, IEEE 41st International Conference on Distributed Computing Systems, DC, USA, pp. 539-549, 2021. https://doi.org/10.1109/ICDCS51616.2021.00058
Sheela, S.; Nataraj, K.R.; Mallikarjunaswamy, S. (2023). A comprehensive exploration of resource allocation strategies within vehicle ad-hoc networks, Mechatronics and Intelligent Transportation Systems, 2(3), pp. 169-190, 2023. https://doi.org/10.56578/mits020305
Shi, T.; Ma, H.; Chen, G.; Hartmann, S. (2020). Location-aware and budget-constrained service deployment for composite applications in multi-cloud environment, IEEE Transactions on Parallel and Distributed Systems, 31(8), pp. 1954-1969, 2020. https://doi.org/10.1109/TPDS.2020.2981306
Smet, P.; Dhoedt, B.; Simoens, P. (2018). Docker layer placement for on-demand provisioning of services on edge clouds, IEEE Transactions on Network and Service Management, 15(3), pp. 1161-1174, 2018. https://doi.org/10.1109/TNSM.2018.2844187
Tan, B.; Ma, H.; Mei, Y. (2020). A NSGA-II-based approach for multi-objective micro-service allocation in container-based clouds, Proc. 20th IEEE/ACM International Symposium on Cluster, Cloud, and Internet Computing, Melbourne, VIC, Australia, pp. 282-289, 2020. https://doi.org/10.1109/CCGrid49817.2020.00-65
Toka, L.; Dobreff, G.; Fodor, B.; Sonkoly, B. (2021). Machine learning based scaling management for Kubernetes edge clusters, IEEE Transactions on Network and Service Management, 18(1), pp. 958-972, 2021. https://doi.org/10.1109/TNSM.2021.3052837
Wang, S.; Guo, Y.; Zhang, N.; Yang, P.; Zhou, A.; Shen, X. (2021). Delay-aware microservice coordination in mobile edge computing: A reinforcement learning approach, IEEE Transactions on Mobile Computing, 20(3), pp. 939-951, 2021. https://doi.org/10.1109/TMC.2019.2957804
Wang, Y. (2025). A Genetic Particle swarm optimization based Hybrid Scheduling Algorithm for Cloud Computing Resources, International Journal of Computers Communications & Control, 20(4), pp. 6814, 2025. https://doi.org/10.15837/ijccc.2025.4.6814
Wang, Z.; O'Boyle, M. (2018). Machine learning in compiler optimization, IEEE Access, 106(11), pp. 1879-1901, 2018. https://doi.org/10.1109/JPROC.2018.2817118
Xie R.; Tang Q.; Wang Q.; Liu X.; Yu F. R.; Huang T. (2020). Satellite-terrestrial integrated edge computing networks: Architecture, challenges, and open issues, IEEE Network, 34(3), pp. 224-231, 2020. https://doi.org/10.1109/MNET.011.1900369
Xu, J.; Chen, L.; Zhou P. (2018). Joint service caching and task offloading for mobile edge computing in dense networks, IEEE International Conference on Computer Communications, Honolulu, HI, USA, pp. 207-215, 2018. https://doi.org/10.1109/INFOCOM.2018.8485977
Yu, Y.; Yang, J.; Guo, C.; Zheng, H.; He, J. (2019). Joint optimization of service request routing and instance placement in the microservice system, Journal of Network and Computer Applications, 147, pp. 102441, 2019. https://doi.org/10.1016/j.jnca.2019.102441
Zhang, X.; Li, Z.; Lai, C.; Zhang J. (2022). Joint edge server placement and service placement in mobile-edge computing, IEEE Internet of Things Journal, 9(13), pp. 11261-11274, 2022. https://doi.org/10.1109/JIOT.2021.3125957
Additional Files
Published
Issue
Section
License
Copyright (c) 2026 Hao Feng, Yunmei Shi, Mingkai Zhen
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
