Enhancing Automated Loading and Unloading of Ship Unloaders through Dynamic 3D Coordinate System with Deep Learning
DOI:
https://doi.org/10.15837/ijccc.2024.2.6234Keywords:
3D coordinates, Convolutional, GAN, Codec, Loading and unloadingAbstract
This paper proposes a deep learning approach for accurate pose estimation in ship unloaders, improving grasping accuracy by reconstructing 3D coordinates. A convolutional neural network optimizes depth map prediction from RGB images, further enhanced by a conditional generative adversarial network to refine quality. Evaluation of simulated ship unloading tasks showed over 90% grasping success rate, outperforming baseline methods. This research offers valuable insights into advanced visual perception and deep learning for next-generation automated cargo handling.
References
Milana, G.; Banisoleiman, K.; González, A. (2021). An investigation into the moving load problem for the lifting boom of a ship unloader, Engineering Structures, 234(1), 111899-111919, 2021.
https://doi.org/10.1016/j.engstruct.2021.111899
Zan, J. (2022). Research on robot path perception and optimization technology based on whale optimization algorithm, Journal of Computational and Cognitive Engineering, 1(4), 201-208, 2022.
https://doi.org/10.47852/bonviewJCCE597820205514
Alam, M. S.; Kwon, K. C.; Kim, N. (2021). Implementation of a Character Recognition System Based on Finger-Joint Tracking Using a Depth Camera, IEEE Transactions on Human-Machine Systems, 51(3), 229-231, 2021.
https://doi.org/10.1109/THMS.2021.3066854
Li, L.; Chen, X.; Zhang, T. (2022). Pose estimation of metal workpieces based on RPM-Net for robot grasping from point cloud, Industrial Robot, 49(6), 1178-1189, 2022.
https://doi.org/10.1108/IR-03-2022-0081
Gao, H. N.; Shen, D. H.; Yu, L. ; Zhang, W. C. (2020). Identification of Cutting Chatter through Deep Learning and Classification, International Journal of Simulation Modelling, 19(4), 667-677.
https://doi.org/10.2507/IJSIMM19-4-CO16
Kushwaha, V.; Shukla, P.; Nandi, G. C. (2023). Generating quality grasp rectangle using Pix2Pix GAN for intelligent robot grasping, Machine Vision and Applications, 34(1), 09821-09836, 2023.
https://doi.org/10.1007/s00138-022-01362-2
Shengqian, L.; Xiaofan, Z. (2022). Research on Hand-eye Calibration Technology of Visual Service Robot Grasping Based on ROS, Chinese Journal of Instrument: English, 9(001), 23-30, 2022.
Higo, R.; Senoo, T.; Ishikawa, M. (2020). Dynamic In-Hand Regrasping Using a High-Speed Robot Hand and High-Speed Vision, IFAC-PapersOnLine, 53(2), 9796-9801, 2020.
https://doi.org/10.1016/j.ifacol.2020.12.2671
Jin, X.; Tang, L.; Li, R.; Zhao, B.; Ji, J.; Ma, Y. (2022). Edge recognition and reduced transplantation loss of leafy vegetable seedlings with Intel RealSense D415 depth camera, Computers and Electronics in Agriculture, 198(1), 107030-107044, 2022.
https://doi.org/10.1016/j.compag.2022.107030
Štampar, M.; Fertalj, K. (2022) Applied Machine Learning in Recognition of DGA Domain Names. Computer Science and Information Systems, 19(1), 205-227, 2022.
https://doi.org/10.2298/CSIS210104046S
He, G.; Huang, X.; Xu, Y.; Qin, J.; Sun, K. (2023). Cable tunnel unmanned aerial vehicle location and navigation method based on ultra wideband and depth camera fusion, IET Electrical Systems in Transportation, 13(1), 12068-12069, 2023.
https://doi.org/10.1049/els2.12068
He, B.; Qian, S.; Niu, Y. (2023). Visual recognition and location algorithm based on optimized YOLOv3 detector and RGB depth camera, The Visual Computer, 240(1), 1-17, 2023.
https://doi.org/10.1007/s00371-023-02895-x
Zhou, C.; Ren, D.; Zhang, X.; Yu, C.; Ju, L. (2022). Human Position Detection Based on Depth Camera Image Information in Mechanical Safety, Advances in Mathematical Physics, 000(1), 9170642-9170652, 2022.
https://doi.org/10.1155/2022/9170642
Zou, L.; Zhuang, K. J.; Zhou, A.; Hu, J. (2023). Bayesian optimization and channel-fusionbased convolutional autoencoder network for fault diagnosis of rotating machinery, Engineering Structures, 280(1), 115708-115720, 2023.
https://doi.org/10.1016/j.engstruct.2023.115708
Cajas, Y. R. A.; Guisado, Y. Z.; Vergaray, A. D. (2023). Identify faults in road structure zones with deep learning. Journal of System and Management Sciences, 13(1), 63-84, 2023.
https://doi.org/10.33168/JSMS.2023.0104
Li, C.; Dong, G.; Wang, R. (2022). A three-dimensional reconstruction algorithm of nonwoven fabric based on an anthill model, Textile Research Journal, 92(11), 1876-1890, 2022.
https://doi.org/10.1177/00405175211073354
Ji, X.; Zhao, Q.; Cheng, J.; Ma, C. (2021). Exploiting spatio-temporal representation for 3D human action recognition from depth map sequences, Knowledge-Based Systems, 227(1), 107040- 107040, 2021.
https://doi.org/10.1016/j.knosys.2021.107040
Ni, J.; Zhou, Z.; Zhao, Y.; Han, Z.; Zhao, L. (2023). Tomato leaf disease recognition based on improved convolutional neural network with attention mechanism, Plant Pathology, 72(7), 1335-1344, 2023.
https://doi.org/10.1111/ppa.13745
Cheng, Y.; Wan, Y.; Sima, Y.; Zhang, Y.; Hu, S. ; Wu, S. (2022). Text Detection of Transformer Based on Deep Learning Algorithm. Tehnički vjesnik, 29(3), 861-866, 2022.
https://doi.org/10.17559/TV-20211027110610
Sun, X.; Zhou, K.; Shi, S.; Song, K.; Chen, X. (2022). A new cyclical generative adversarial network based data augmentation method for multiaxial fatigue life prediction, International Journal of Fatigue, 162(1), 106996-107013, 2022.
https://doi.org/10.1016/j.ijfatigue.2022.106996
Saeed, A.; Hayat, M. F.; Habib, T.; Ghaffar, D. A.; Qureshi, M. A. (2022). A novel multi-speakers Urdu singing voices synthesizer using Wasserstein Generative Adversarial Network, Speech Communication, 137(1), 103-113, 2022.
https://doi.org/10.1016/j.specom.2021.12.005
Zhang, H.; Chen, X.; Lu, S.; Yao, L.; Chen, X. (2023). A contrastive learning-based attention generative adversarial network for defect detection in color-patterned fabric, Coloration Technology, 139(3), 248-264, 2023.
https://doi.org/10.1111/cote.12642
Wang, M.; Luo, J.; Zheng, L.; Yuan, J.; Walter, U. (2020). Generate optimal grasping trajectories to the end-effector using an improved genetic algorithm, Advances in Space Research, 66(7), 1803- 1817, 2020.
https://doi.org/10.1016/j.asr.2020.06.022
Hong, L. C.; Tee, C.; Goh, M. K. O. (2022). Activities of daily living recognition using deep learning approaches. Journal of Logistics, Informatics and Service Science, 9(4), 129-148, 2022.
Jiang, C.; Wang, D.; Zhao, B.; Liao, Z.; Gu, G. (2021). Modeling and inverse design of bioinspired multi-segment pneu-net soft manipulators for 3D trajectory motion, Applied Physics Reviews, 8(4), 054468-054468, 2021.
Additional Files
Published
Issue
Section
License
Copyright (c) 2023 Shijie Sun, Lufeng Wang, Qu Li, Wei Fu, Fei Jiang, Tianxing Song, Guangbo Pi
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
