Human-in-the-loop: Role in Cyber Physical Agricultural Systems
Keywords:
Agricultural Robotics, Collaborative Intelligence, Human-in-the-Loop, Multi-Agent SimulationAbstract
With increasing automation, the ‘human’ element in industrial systems is gradually being reduced, often for the sake of standardization. Complete automation, however, might not be optimal in complex, uncertain environments due to the dynamic and unstructured nature of interactions. Leveraging human perception and cognition can prove fruitful in making automated systems robust and sustainable. “Human-in-the-loop” (HITL) systems are systems which incorporate meaningful human interactions into the workflow. Agricultural Robotic Systems (ARS), developed for the timely detection and prevention of diseases in agricultural crops, are an example of cyber-physical systems where HITL augmentation can provide improved detection capabilities and system performance. Humans can apply their domain knowledge and diagnostic skills to fill in the knowledge gaps present in agricultural robotics and make them more resilient to variability. Owing to the multi-agent nature of ARS, HUB-CI, a collaborative platform for the optimization of interactions between agents is emulated to direct workflow logic. The challenge remains in designing and integrating human roles and tasks in the automated loop. This article explains the development of a HITL simulation for ARS, by first realistically modeling human agents, and exploring two different modes by which they can be integrated into the loop: Sequential, and Shared Integration. System performance metrics such as costs, number of tasks, and classification accuracy are measured and compared for different collaboration protocols. The results show the statistically significant advantages of HUB-CI protocols over the traditional protocols for each integration, while also discussing the competitive factors of both integration modes. Strengthening human modeling and expanding the range of human activities within the loop can help improve the practicality and accuracy of the simulation in replicating a HITL-ARS.
References
[2] Aqeel-Ur-Rehman, Abu Zafar Abbasi, Noman Islam, and Zubair Ahmed Shaikh. 2014. "A Review of Wireless Sensors and Networks' Applications in Agriculture." Computer Standards and Interfaces 36 (2): 263-70, 2014. https://doi.org/10.1016/j.csi.2011.03.004
[3] Bechar, Avital, and Yael Edan. 2003. "Human-Robot Collaboration for Improved Target Recognition of Agricultural Robots." Industrial Robot30 (5): 432-36. https://doi.org/10.1108/01439910310492194
[4] Bechar, Avital, Yael Edan, and Joachim Meyer. 2006. "Optimal Collaboration in Human-Robot Target Recognition Systems." Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics 5: 4243-48, 2006. https://doi.org/10.1109/ICSMC.2006.384801
[5] Bechar, Avital, Joachim Meyer, and Yael Edan. 2007. "An Objective Function to Evaluate Performance of Human-Robot Systems for Target Recognition Tasks." Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics 84105: 967-72, 2007. https://doi.org/10.1109/ICSMC.2007.4414015
[6] Behmann, Jan, Jí¶rg Steinrücken, and Lutz Plümer. 2014. "Detection of Early Plant Stress Responses in Hyperspectral Images." ISPRS Journal of Photogrammetry and Remote Sensing 93: 98-111, 2014. https://doi.org/10.1016/j.isprsjprs.2014.03.016
[7] Belforte, G.; Deboli, Gay, R. P.; Piccarolo,P.; Ricauda Aimonino, D. (2006). "Robot Design and Testing for Greenhouse Applications." Biosystems Engineering 95 (3): 309-21, 2006. https://doi.org/10.1016/j.biosystemseng.2006.07.004
[8] Blanchet, Max, Thomas Rinn, Georg Von Thaden, and G De Thieulloy. 2014. "Industry 4.0: The New Industrial Revolution-How Europe Will Succeed." Hg. v. Roland Berger Strategy Consultants GmbH. München. Abgerufen Am 11.05. 2014, Unter Http://Www. Rolandberger. Com/Media/Pdf/Roland_Berger_TAB_Industry_4_0_2014 0403. Pdf.
[9] Branson, Steve, Catherine Wah, Florian Schroff, Boris Babenko, Peter Welinder, Pietro Perona, and Serge Belongie. 2010. "Visual Recognition with Humans in the Loop." In Proceedings of the 11th European Conference on Computer Vision: Part IV, 438-51. ECCV'10. Berlin, Heidelberg: Springer-Verlag. http://dl.acm.org/citation.cfm?id=1888089.1888123. https://doi.org/10.1007/978-3-642-15561-1_32
[10] Bruno, Giulia, and Dario Antonelli. 2018. "Dynamic Task Classification and Assignment for the Management of Human-Robot Collaborative Teams in Workcells." The International Journal of Advanced Manufacturing Technology 98 (9): 2415-27. https://doi.org/10.1007/s00170-018-2400- 4.
[11] Cai, Hong, and Yasamin Mostofi. 2019. "Human-Robot Collaborative Site Inspection under Resource Constraints." IEEE Transactions on Robotics 35 (1): 200-215. https://doi.org/10.1109/TRO.2018.2875389.
[12] Devadasan, Prabhu, Hao Zhong, and Shimon Y. Nof. 2013. "Collaborative Intelligence in Knowledge Based Service Planning." Expert Systems with Applications 40 (17): 6778-87. https://doi.org/10.1016/j.eswa.2013.06.048.
[13] Durugbo, Christopher. 2016. "Collaborative Networks: A Systematic Review and Multi- Level Framework." International Journal of Production Research 54 (12): 3749-76. https://doi.org/10.1080/00207543.2015.1122249.
[14] Dusadeerungsikul, Puwadol Oak, and Shimon Y. Nof. 2019. "A Collaborative Control Protocol for Agricultural Robot Routing with Online Adaptation." Computers and Industrial Engineering 135 (August 2018): 456-66. https://doi.org/10.1016/j.cie.2019.06.037.
[15] Dusadeerungsikul, Puwadol Oak, Shimon Y Nof, Avital Bechar, and Yang Tao. 2019. "Collaborative Control Protocol for Agricultural Cyber-Physical System." Procedia Manufacturing 39: 235-42. https://doi.org/https://doi.org/10.1016/j.promfg.2020.01.330.
[16] Dusadeerungsikul, Puwadol Oak, Maitreya Sreeram, Xiang He, Ashwin Nair, Karthik Ramani, Alexander J. Quinn, and Shimon Y. Nof. 2019. "Collaboration Requirement Planning Protocol for Hub-Ci in Factories of the Future." Procedia Manufacturing 39: 218-25. https://doi.org/10.1016/j.promfg.2020.01.327.
[17] Fang, Yi, and Ramaraja P Ramasamy. 2015. "Current and Prospective Methods for Plant Disease Detection." Biosensors 5 (3): 537-61. https://doi.org/10.3390/bios5030537.
[18] Gongal, A., S. Amatya, M. Karkee, Q. Zhang, and K. Lewis. 2015. "Sensors and Systems for Fruit Detection and Localization: A Review." Computers and Electronics in Agriculture 116: 8-19. https://doi.org/10.1016/j.compag.2015.05.021.
[19] Guo, Ping, Puwadol Dusadeeringsikul, and Shimon Y. Nof. 2018. "Agricultural Cyber Physical System Collaboration for Greenhouse Stress Management." Computers and Electronics in Agriculture 150 (June 2017): 439-54. https://doi.org/10.1016/j.compag.2018.05.022.
[20] Holzinger, Andreas, Markus Plass, Michael Kickmeier-rust Katharina, Holzinger Gloria, and Cerasela Cris. 2019. "Interactive Machine Learning: Experimental Evidence for the Human in the Algorithmic Loop A Case Study on Ant Colony Optimization," 2401-14. https://doi.org/10.1007/s10489-018-1361-5
[21] Jaber, M. Y., and W. P. Neumann. 2010. "Modelling Worker Fatigue and Recovery in Dual-Resource Constrained Systems." Computers and Industrial Engineering 59 (1): 75-84. https://doi.org/10.1016/j.cie.2010.03.001.
[22] Lee, Edward A. 2010. "CPS Foundations." Proceedings - Design Automation Conference, 737-42. https://doi.org/10.1145/1837274.1837462.
[23] Lentz, E C, Compendium Project, B L Gardner, G C Rausser, M T Ruel, Food Security Assessment, Food Security Statistics, et al. 2010. "References and Notes 1.," no. February.
[24] Mclennan, Michael, and Rick Kennell. 2010. "HUBzero: A Platform for Dissemination and Collaboration in Computational Science and Engineering." Computing in Science and Engineering 12 (2): 48-52. https://doi.org/10.1109/MCSE.2010.41.
[25] Munir, S., Stankovic, J. a, Liang, C.-J. M., Lin, S. 2013. "Cyber Physical System Challenges for Human-in-the-Loop Control." The 8th International Workshop on Feedback Computing.
[26] Nair, A S, A Bechar, Y Tao, and S Y Nof. 2019. "The HUB-CI Model for Telerobotics in Greenhouse Monitoring." Procedia Manufacturing 39: 414-21. https://doi.org/10.1016/j.promfg.2020.01.385
[27] Nguyen, Win P V, and Shimon Y Nof. 2019. "Collaborative Response to Disruption Propagation (CRDP) in Cyber-Physical Systems and Complex Networks." Decision Support Systems 117: 1-13. https://doi.org/10.1016/j.dss.2018.11.005
[28] Nof, S. Y. 2007. "Collaborative Control Theory for E-Work, e-Production, and e-Service." Annual Reviews in Control 31 (2): 281-92. https://doi.org/10.1016/j.arcontrol.2007.08.002.
[29] Rad, Ciprian-Radu, Olimpiu Hancu, Ioana-Alexandra Takacs, and Gheorghe Olteanu. 2015. "Smart Monitoring of Potato Crop: A Cyber-Physical System Architecture Model in the Field of Precision Agriculture." Agriculture and Agricultural Science Procedia 6: 73-79. https://doi.org/10.1016/j.aaspro.2015.08.041
[30] Ramin Shamshiri, Redmond, Ibrahim A. Hameed, Lenka Pitonakova, Cornelia Weltzien, Siva K. Balasundram, Ian J. Yule, Tony E. Grift, and Girish Chowdhary. 2018. "Simulation Software and Virtual Environments for Acceleration of Agricultural Robotics: Features Highlights and Performance Comparison." International Journal of Agricultural and Biological Engineering 11 (4): 12-20. https://doi.org/10.25165/j.ijabe.20181104.4032.
[31] Schirner, Gunar, Deniz Erdogmus, Kaushik Chowdhury, and Taskin Padir. 2013. "The Future of Human-in-the-Loop Cyber-Physical Systems." Computer 46 (1): 36-45. https://doi.org/10.1109/MC.2013.31.
[32] Sheridan, Thomas B, and William L Verplank. 1978. "Human and Computer Control of Undersea Teleoperators." https://doi.org/10.21236/ADA057655
[33] Sreeram, Maitreya. 2020. "Human-in-the-Loop of Cyber Physical Agricultural Robotic Systems." Purdue University Graduate School. https://doi.org/10.25394/PGS.13303490.v1.
[34] Stewart, A, M Cao, A Nedic, D Tomlin, and N Leonard. 2012. "Towards Human- Robot Teams: Model-Based Analysis of Human Decision Making in Two-Alternative Choice Tasks With Social Feedback." Proceedings of the IEEE 100 (3): 751-75. https://doi.org/10.1109/JPROC.2011.2173815.
[35] Wang, Dongyi, Robert Vinson, Maxwell Holmes, Gary Seibel, Avital Bechar, Shimon Nof, and Yang Tao. 2019. "Early Detection of Tomato Spotted Wilt Virus by Hyperspectral Imaging and Outlier Removal Auxiliary Classifier Generative Adversarial Nets (OR-AC-GAN)." Scientific Reports 9 (1): 4377. https://doi.org/10.1038/s41598-019-40066-y.
[36] Winkelhaus, S., F. Sgarbossa, M. Calzavara, and E. H. Grosse. 2018a. "The Effects of Human Fatigue on Learning in Order Picking: An Explorative Experimental Investigation." IFACPapersOnLine 51 (11): 832-37. https://doi.org/10.1016/j.ifacol.2018.08.442.
[37] Zhong, H.; Levalle,R.R.; Mohsen Moghaddam, Nof, S.Y. 2015. "Collaborative Intelligence - Definition and Measured Impacts on Internetworked e-Work." Management and Production Engineering Review 6 (1): 67-78, 2015. https://doi.org/10.1515/mper-2015-0009
[38] Zhong, H.;, Wachs,J.P.; Nof, S.Y. 2014. "Telerobot-Enabled HUB-CI Model for Collaborative Lifecycle Management of Design and Prototyping." Computers in Industry 65 (4): 550-62, 2014. https://doi.org/10.1016/j.compind.2013.12.011
Additional Files
Published
Issue
Section
License
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
