An improved ant colony algorithm based on Q-Learning for route planning of autonomous vehicle
DOI:
https://doi.org/10.15837/ijccc.2024.3.5382Keywords:
Autonomous vehicle, Path planning, Q-Learning, Improved ant colony algorithmAbstract
In view of the problems existing in the path planning algorithms of unmanned vehicles, such as low search efficiency, slow convergence speed and easy to fall into the local optimal. Based on the characteristics of route planning for unmanned vehicles, this paper introduces Q-Learning into the traditional ant colony algorithm to enhance the learning ability of the algorithm in dynamic environment, so as to improve the overall efficiency of route search. By mapping pheromones into Q values in Q-learning, rapid search in complex environments is realized, and a collection-free path satisfying constraints is quickly found. The results of case analysis show that compared with the traditional ant colony algorithm and the improved ant colony algorithm considering reward and punishment factors, the improved ant colony algorithm based on Q-Learning can effectively reduce the number of iterations, shorten the path optimization time and path length and other performance indicators, and has many advantages in jumping out of the local optimal, improving the global search ability and improving the convergence speed, and has good adaptability and robustness in complex environments. It ensures the safety and stability of unmanned vehicles in complex environments.
References
Katrakazas C; Quddus M; Chen W H; et al. (2015). Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions, Transportation Research Part C, 60, 416-442, 2015.
https://doi.org/10.1016/j.trc.2015.09.011
Patle B; Pandey A; Parhi D; et al. (2019). A review: On path planning strategies for navigation of mobile robot, Defence Technology, 15(4), 582-606, 2019.
https://doi.org/10.1016/j.dt.2019.04.011
LI T C; SUN S D; GAO Y. (2010). Fan-shaped Grid Based Global Path Planning for Mobile Robot, ROBOT, 32(4), 547-552, 2010.
https://doi.org/10.3724/SP.J.1218.2010.00547
GUO L J; SHI W X; LI Y; LI F X; et al. (2011). Mapping algorithm using adaptive size of occupancy grids based on quadtree, Control and Decision, 26(11), 1690-1694, 2011.
Y Yang; K He; Y P Wang; Z Z Yuan; Y H Yin; M Z Guo. (2022). Identification of dynamic traffic crash risk for cross-area freeways based on statistical and machine learning methods, Physica A: Statistical Mechanics and its Applications, 595(2022), 127083, 2022.
https://doi.org/10.1016/j.physa.2022.127083
Azim E; Chaoxian W; Chuanyang S. (2021). Research Advances and Challenges of Autonomous and Connected Ground Vehicles, IEEE Transactions on Intelligent Transportation Systems, 22(2), 683-711, 2021.
https://doi.org/10.1109/TITS.2019.2958352
LI D L,WANG P, DU L. (2019). Path planning technologies for autonomous underwater vehicles-a review, IEEE Access, 7, 9745-9768, 2019.
https://doi.org/10.1109/ACCESS.2018.2888617
Özgur C; Sarikovanlik V. (2022). Forecasting BIST100 and NASDAQ Indices with Single and Hybrid Machine Learning Algorithms, Economic Computation And Economic Cybernetics Studies And Research, DOI: 10.24818/18423264/56.3.22.15, 56(3), 235-250, 2022.
https://doi.org/10.24818/18423264/56.3.22.15
Y. Yang; N. Tian; Y. Wang; Z. Yuan. (2022). A Parallel FP-Growth Mining Algorithm with Load Balancing Constraints for Traffic Crash Data, International Journal of Computers Communications & Control, 17(4), 4806, 2022.
https://doi.org/10.15837/ijccc.2022.4.4806
Liu J.-Y.; Liu S.-F.; Gong D.-Q. (2021). Electric Vehicle Charging Station Layout Based on Particle Swarm Simulation, Int. Journal of Simulation Modelling, 20(4), 754-765, 2021.
https://doi.org/10.2507/IJSIMM20-4-CO17
Yang Y; Yuan Z; Meng R. (2022). Exploring Traffic Crash Occurrence Mechanism toward Cross- Area Freeways via an Improved Data Mining Approach, Journal of Transportation Engineering Part A Systems, 148(9), 04022052, 2022.
https://doi.org/10.1061/JTEPBS.0000698
Bacha A; Bauman C; Faruque R; et al. (2008). Odin: Team Victor Tango's entry in the DARPA Urban Challenge, Journal of Field Robotics, 25(8), 467-92, 2008.
https://doi.org/10.1002/rob.20248
Zhang X Y; Zou Y S. (2021). Collision-free path planning for automated guided vehicles based on improved A* algorithm, Systems Engineering-Theory & Practice, 41(1), 240-246, 2021.
Carreras M; Hernandez J D; Vidal E; et al. (2016). Online motion planning for underwater inspection, Autonomous Underwater Vehicles. IEEE, 336-341, 2016.
https://doi.org/10.1109/AUV.2016.7778693
Jalalmaab M; Fidan B; Jeon S; et al. (2015). Model predictive path planning with timevarying safety constraints for highway autonomous driving, International Conference on Advanced Robotics (ICAR), 213-217, 2015.
https://doi.org/10.1109/ICAR.2015.7251458
Receveur J-B; Victor S; Melchior P. (2020). Autonomous car decision making and trajectory tracking based on genetic algorithms and fractional potential fields, Intelligent Service Robotics, 13(2), 315-330, 2020.
https://doi.org/10.1007/s11370-020-00314-x
Afify, H.M.; Mohammed, K.K.; Hassanien, A.E. (2020). Multi-Images Recognition of Breast Cancer Histopathological via Probabilistic Neural Network Approach, Journal of System and Management Sciences, 10(2), 53-68, 2020.
https://doi.org/10.33168/JSMS.2020.0204
Miao C W; Chen G Z; Yan C L; et al. (2021). Path planning optimization of indoor mobile robot based on adaptive ant colony algorithm, Computers & Industrial Engineering, 156(1), 1-12, 2021.
https://doi.org/10.1016/j.cie.2021.107230
XU L; FU W H; JIANG W H; LI Z T. (2021). mobile robots path planning based on 16-directions 24-neighborhoods improved ant colony algorithm, Control and Decision, 36(05), 1137-1146, 2021.
LI T; ZHAO H S. (2022). Path optimization for mobile robot based on evolutionary ant colony algorithm, Control and Decision, DOI:10.13195/j.kzyjc.2021.1324, 1-9, 2022.
LI S D; XU X; ZUO L. (2015). Dynamic path planning of a mobile robot with improved Qlearning algorithm, In Proceedings of 2015 IEEE International Conference on Information and Automation, 409-414, 2015.
https://doi.org/10.1109/ICInfA.2015.7279322
Yang Y; Yuan Z; Chen J; Guo M. (2017). Assessment of osculating value method based on entropy weight to transportation energy conservation and emission reduction, Environmental Engineering & Management Journal, 16(10), 2413-2424, 2017.
https://doi.org/10.30638/eemj.2017.249
Yang Y; Yang B; Yuan Z; et al. (2023). Modeling and Comparing Two Modes of Sharing Parking Spots at Residential Area: Real-time and Fixed-time Allocation, IET Intelligent Transport Systems, 2023.
https://doi.org/10.1049/itr2.12343
Yuan Z; Yuan X; Yang Y; et al. (2023). Greenhouse Gas Emission Analysis and Measurement for Urban Rail Transit: A Review of Research Progress and Prospects, Digital Transportation and Safety, 1(1), 37-52, 2023.
https://doi.org/10.48130/DTS-2023-0004
Tan B; Peng Y Y; Lin J G. (2021). A local path planning method based on q-learning, In International Conference on Signal Processing and Machine Learning, 80-84, 2021.
https://doi.org/10.1109/CONF-SPML54095.2021.00024
TIAN X H; HUO X; ZHOU D L; ZHAO H. (2022). Ant colony pheromone aided Q-learning path planning algorithm, Control and Decision, DOI: https://doi.org/10.13195/j.kzyjc.2022.0476, 2022.
MEERZA S I A; ISLAM M; UZZAL M M. (2019). Q-learning based particle swarm optimization algorithm for optimal path planning of swarm of mobile robots, Proceedings of 2019 International Conference on Advances in Science, Engineering and Robotics Technology, 1-5, 2019.
https://doi.org/10.1109/ICASERT.2019.8934450
YAO Q F; ZHENG Z Y; QI L; et al. (2020). Path planning method with improved artificial potential field- a reinforcement learning perspective, IEEE Access, 8, 135513-135523, 2020.
https://doi.org/10.1109/ACCESS.2020.3011211
LIU Z Y; LAN F; YANG H B. (2019). Partition heuristic RRT algorithm of path planning based on Q-learning, Proceedings of 2019 Advanced Information Technology, Electronic and Automation Control Conference, 386-392, 2019.
https://doi.org/10.1109/IAEAC47372.2019.8997878
SHI Z G; TU J; ZHANG Q; et al. (2013). The improved Q-Learning algorithm based on pheromone mechanism for swarm robot system, Proceedings of the 32nd Chinese Control Conference, 6033- 6038, 2013.
Zhu J Y; GAO M T. (2021). AUV Path Planning Based on Particle Swarm Optimization and Improved Ant Colony Optimization, Computer Engineering and Applications, 57(06), 267-273, 2021.
HU C Y; JIANG P; ZHOU G R. (2020). Application of improved ant colony algorithm in AGV path planning, Computer Engineering and Applications, 56(8), 270-278, 2020.
MA Y N; GONG Y J; XIAO C F; et al. (2019). Path planning for autonomous underwater vehicles: an ant colony algorithm incorporating alarm pheromone, IEEE Transactions on Vehicular Technology, 68(1), 141-154, 2019.
https://doi.org/10.1109/TVT.2018.2882130
HE X L; JIANG H; SONG Y; et al. (2019). Routing selection with reinforcement learning for energy harvesting multi-hop CRN, IEEE Access, 7, 54435-54448, 2019.
https://doi.org/10.1109/ACCESS.2019.2912996
ARUNITA K; LOBIYAL D K. (2021). Q-learning based routing protocol to enhance network lifetime in WSNs, International Journal of Computer Networks & Communications, 13(2), 67- 80, 2021.
Additional Files
Published
Issue
Section
License
Copyright (c) 2024 Liping Zhao, Feng Li, Dongye Sun, Zihan Zhao
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
