Urban Noise Analysis and Emergency Detection System using Lightweight End-to-End Convolutional Neural Network
DOI:
https://doi.org/10.15837/ijccc.2023.5.5814Keywords:
End-to-end neural network, Environmental sound classification, FIWARE, On-device deep learningAbstract
In recent years, the application of deep learning to environmental sound classification (ESC) has received considerable attention owing to its powerful ability to recognize the context of urban sounds. In general, deep learning models with high accuracy require substantial computing and memory resources. Consequently, to apply complex deep learning models to ESC in the real world, model inference has been performed on cloud servers with powerful computing resources. However, heavy network traffic and security issues occur when inferences are performed on a cloud server. In addition, deploying a deep learning model trained on a single public ESC dataset may not be sufficient for classifying various classes of urban noise and emergency-related sounds. To address these problems, we propose an on-device, real-time urban sound monitoring system that can classify various urban sounds at low system construction costs. The proposed system consisted of an edge artificial intelligence (AI) node and a FIWARE-based server. To enable the real-time inference on a resource-constrained edge AI node, we developed a lightweight convolutional neural network (CNN) by adjusting the input and model configurations to achieve high accuracy with a low number of parameters. The model achieved 94.9% classification accuracy using only 331 K parameters on an integrated dataset that included 17 classes of urban noises and emergencies. Furthermore, a prototype of the proposed system was developed and evaluated to verify its feasibility. The prototype system was built at a cost of less than USD 50 and could perform the entire monitoring process every 3 s. We also visualized the sound monitoring data using Grafana on a FIWARE-based server.References
Abdoli, S.; Cardinal, P.; Koerich, A.L. (2019). End-to-end environmental sound classification using a 1D convolutional neural network, Expert Systems with Applications, 136, 252-263, 2019.
https://doi.org/10.1016/j.eswa.2019.06.040
Ahn, H.; Chen, T.; Alnaasan, N.; Shafi, A.; Abduljabbar, M.; Subramoni, H.; Panda, D.K.
(2023). Performance characterization of using quantization for DNN inference on edge devices: Extended version, arXiv:2303.05016, 2023.
Almaadeed, N.; Asim, M.; Al-Maadeed, S.; Bouridane, A.; Beghdadi, A. (2018). Automatic detection and classification of audio events for road surveillance applications, Sensors, 18(6), 1858, 2018.
https://doi.org/10.3390/s18061858
Arce, P.; Salvo, D.; Piñero, G.; Gonzalez, A. (2021). FIWARE based low-cost wireless acoustic sensor network for monitoring and classification of urban soundscape, Computer Networks, 196, 108199, 2021.
https://doi.org/10.1016/j.comnet.2021.108199
Asdrubali, F.; D'Alessandro, F. (2018). Innovative approaches for noise management in smart cities: A review, Current Pollution Reports, 4(2), 143-153, 2018.
https://doi.org/10.1007/s40726-018-0090-z
Chachada, S.; Kuo, C.-C.J. (2014). Environmental sound recognition: A survey, APSIPA Transactions on Signal and Information Processing, 3, e14, 2014.
https://doi.org/10.1017/ATSIP.2014.12
Cirillo, F.; Solmaz, G.; Berz, E.L.; Bauer, M.; Cheng, B.; Kovacs, E. (2019). A standard-based open source IoT platform: FIWARE, IEEE Internet of Things Magazine, 2(3), 12-18, 2019.
https://doi.org/10.1109/IOTM.0001.1800022
da Silva, B.; Happi, A.W.; Braeken, A.; Touhafi, A. (2019). Evaluation of classical machine learning techniques towards urban sound recognition on embedded systems, Applied Sciences, 9(18), 3885, 2019.
https://doi.org/10.3390/app9183885
Elliott, D.; Otero, C.E.; Wyatt, S.; Martino, E. (2021). Tiny transformers for environmental sound classification at the edge, arXiv:2103.12157, 2021.
Fazio, M.; Celesti, A.; Márquez, F.G.; Glikson, A.; Villari, M. (2015). Exploiting the FIWARE cloud platform to develop a remote patient monitoring system, 2015 IEEE Symposium on Computers and Communications (ISCC), 264-270, 2015.
https://doi.org/10.1109/ISCC.2015.7405526
Gazneli, A.; Zimerman, G.; Ridnik, T.; Sharir, G.; Noy, A. (2022). End-to-end audio strikes back: Boosting augmentations towards an efficient audio classification network, arXiv:2204.11479, 2022.
Gemmeke, J.F.; Ellis, D.P.W.; Freedman, D.; Jansen, A.; Lawrence, W.; Moore, R.C.; Plakal, M.; Ritter, M. (2017). Audio Set: An ontology and human-labeled dataset for audio events, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 776-780, 2017.
https://doi.org/10.1109/ICASSP.2017.7952261
Guzhov, A.; Raue, F.; Hees, J.; Dengel, A. (2021). ESResNet: Environmental sound classification based on visual domain models, Proceedings of 2020 25th International Conference on Pattern Recognition (ICPR), 4933-4940, 2021.
https://doi.org/10.1109/ICPR48806.2021.9413035
Han, S.; Pool, J.; Tran, J.; Dally, W. (2015). Learning both weights and connections for efficient neural network, Advances in Neural Information Processing Systems (NIPS), 28, 2015.
Hinton, G.; Vinyals, O.; Dean, J. (2015). Distilling the knowledge in a neural network, arXiv:1503.02531, 2015.
Ioffe, S.; Szegedy, C. (2015). Batch Normalization: Accelerating deep network training by reducing internal covariate shift, Proceedings of the 32nd International Conference on Machine Learning, 37, 448-456, 2015.
Li, S.; Yao, Y.; Hu, J.; Liu, G.; Yao, X.; Hu, J. (2018). An ensemble stacked convolutional neural network model for environmental event sound recognition, Applied Sciences, 8(7), 1152, 2018.
https://doi.org/10.3390/app8071152
López-Riquelme, J.A.; Pavón-Pulido, N.; Navarro-Hellín, H.; Soto-Valles, F.; Torres-Sánchez, R. (2017). A software architecture based on FIWARE cloud for precision agriculture, Agricultural Water Management, 182, 123-135, 2017.
https://doi.org/10.1016/j.agwat.2016.10.020
Maijala, P.; Shuyang, Z.; Heittola, T.; Virtanen, T. (2018). Environmental noise monitoring using source classification in sensors, Applied Acoustics, 129, 258-267, 2018.
https://doi.org/10.1016/j.apacoust.2017.08.006
Murshed, M.G.S.; Murphy, C.; Hou, D.; Khan, N.; Ananthanarayanan, G.; Hussain, F. (2021). Machine learning at the network edge: A survey, ACM Computing Surveys, 54(8), 1-37, 2021.
https://doi.org/10.1145/3469029
Padhy, S.; Tiwari, J.; Rathore, S.; Kumar, N. (2019). Emergency signal classification for the hearing impaired using multi-channel convolutional neural network architecture, Proceedings of 2019 IEEE Conference on Information and Communication Technology, 1-6, 2019.
https://doi.org/10.1109/CICT48419.2019.9066252
Piczak, K.J. (2015). Environmental sound classification with convolutional neural networks, Proceedings of 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), 1-6, 2015.
https://doi.org/10.1109/MLSP.2015.7324337
Piczak, K.J. (2015). ESC: Dataset for environmental sound classification, Proceedings of the 23rd ACM International Conference on Multimedia, 1015-1018, 2015.
https://doi.org/10.1145/2733373.2806390
Salamon, J.; Jacoby, C.; Bello, J.P. (2014). A dataset and taxonomy for urban sound research, Proceedings of the 22nd ACM International Conference on Multimedia, 1041-1044, 2014.
https://doi.org/10.1145/2647868.2655045
Salamon, J.; Bello, J.P. (2017). Deep convolutional neural networks and data augmentation for environmental sound classification, IEEE Signal Processing Letters, 24(3), 279-283, 2017.
https://doi.org/10.1109/LSP.2017.2657381
Segura-Garcia, J.; Felici-Castell, S.; Perez-Solano, J.J.; Cobos, M.; Navarro, J.M. (2015). Low-cost alternatives for urban noise nuisance monitoring using wireless sensor networks, IEEE Sensors Journal, 15(2), 836-844, 2015.
https://doi.org/10.1109/JSEN.2014.2356342
Stansfeld, S.A.; Matheson M.P. (2003). Noise pollution: Non-auditory effects on health, British Medical Bulletin, 68(1), 243-257, 2003.
https://doi.org/10.1093/bmb/ldg033
Tanweer, S.; Mobin, A.; Alam, A. (2016). Environmental noise classification using LDA, QDA, and ANN methods, Indian Journal of Science and Technology, 9(33), 1-8, 2016.
https://doi.org/10.17485/ijst/2016/v9i33/95628
Tran, V.-T.; Tsai, W.-H. (2020). Acoustic-based emergency vehicle detection using convolutional neural networks, IEEE Access, 8, 75702-75713, 2020.
https://doi.org/10.1109/ACCESS.2020.2988986
Tsalera, E.; Papadakis, A.; Samarakou, M. (2020). Monitoring, profiling, and classification of urban environmental noise using sound characteristics and the KNN algorithm, Energy Reports, 6, 223-230, 2020.
https://doi.org/10.1016/j.egyr.2020.08.045
World Health Organization (2018). Environmental Noise Guidelines for the European Region, World Health Organization. Regional Office for Europe, 2018.
Wu, J.; Leng, C.; Wang, Y.; Hu, Q.; Cheng, J. (2016). Quantized convolutional neural networks for mobile devices, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4820-4828, 2016.
https://doi.org/10.1109/CVPR.2016.521
Wu, H.; Judd, P.; Zhang, X.; Isaev, M.; Micikevicius, P. (2020). Integer quantization for deep learning inference: Principles and empirical evaluation, arXiv:2004.09602, 2020.
Wyatt, S.; Elliott, D.; Aravamudan, A.; Otero, C.E.; Otero, L.D.; Anagnostopoulos, G.C.; Smith, A.O.; Peter, A.M.; Jones, W.; Leung, S.; Lam, E. (2021). Environmental sound classification with tiny transformers in noisy edge environments, 2021 IEEE 7th World Forum on Internet of Things (WF-IoT), 309-314, 2021.
https://doi.org/10.1109/WF-IoT51360.2021.9596007
Zhang, H.; McLoughlin, I.; Song, Y. (2015). Robust sound event recognition using convolutional neural networks, Proceedings of 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 559-563, 2015.
https://doi.org/10.1109/ICASSP.2015.7178031
Zyrianoff, I.; Heideker, A.; Silva, D.; Kamienski, C. (2018). Scalability of an Internet of Things platform for smart water management for agriculture, 2018 23rd Conference of Open Innovations Association (FRUCT), 432-439, 2018.
https://doi.org/10.23919/FRUCT.2018.8588086
Grafana, [Online]. Available: https://grafana.com, Accessed on 23 June 2023.
Natural and Artificial Occurrence Nonverbal Sound Datasets, [Online]. Available: https://aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&aihubDataSe= realm&dataSetSn=644, Accessed on 23 June 2023.
Post-training quantization, [Online]. Available: https://www.tensorflow.org/lite/ performance/post_training_quantization#optimization_methods, Accessed on 23 June 2023.
Urban Sound Dataset, [Online]. Available: https://aihub.or.kr/aihubdata/data/view.do? currMenu=115&topMenu=100&aihubDataSe=realm&dataSetSn=585, Accessed on 23 June 2023.
Additional Files
Published
Issue
Section
License
Copyright (c) 2023 Jinho Park, Taeyoung Yoo, Seongjae Lee, Taehyoun Kim
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
