(Multimedia University, Malaysia)(2) * Tee Connie
(Multimedia University, Malaysia)(3) Thian Song Ong
(Multimedia University, Malaysia)(4) Andrew Beng Jin Teoh
(Yonsei University, Korea, Democratic People's Republic of)(5) Pin Shen Teh
(Manchester Metropolitan University, United Kingdom)*corresponding author
AbstractIn response to the critical need for enhanced public safety measures, this study introduces an advanced intelligent surveillance system designed to autonomously detect abnormal behaviors within public spaces. Leveraging the computational efficiency and accuracy of a Simple Recurrent Unit (SRU) integrated with an attention mechanism, this research delivers a novel approach towards understanding and interpreting human interactions in real-time video footage. Distinctively, the model specializes in identifying two primary categories of abnormal behavior: aggressive two-person interactions such as physical confrontations and collective crowd dynamics, characterized by sudden dispersal patterns indicative of distress or danger. The incorporation of Attention mechanism precisely targets critical elements of behavior, thereby enhancing the model's focus and interpretative clarity. Empirical validation across five benchmark datasets reveals that our model not only outperforms traditional Long Short-Term Memory (LSTM) frameworks in terms of speed by a factor of 1.5 but also demonstrates superior accuracy in abnormal behavior recognition. These findings not only underscore the model's potential in preempting potential safety threats but also mark a significant advancement in the application of deep learning technologies for public security infrastructures. This research contributes to the broader discourse on public safety, offering actionable insights and robust technological solutions to enhance surveillance efficacy and response mechanisms in critical public domains.
KeywordsAbnormal Behavior Recognition;Simple Recurrent Unit;Attention Mechanism;Long Short-Term Memory
|
DOIhttps://doi.org/10.26555/ijain.v10i2.1385 |
Article metricsAbstract views : 275 | PDF views : 66 |
Cite |
Full Text Download
|
References
[1] S. Roka, M. Diwakar, P. Singh, and P. Singh, “Anomaly behavior detection analysis in video surveillance: a critical review,” J. Electron. Imaging, vol. 32, no. 04, p. 042106, Mar. 2023, doi: 10.1117/1.JEI.32.4.042106.
[2] M. Cho, T. Kim, W. J. Kim, S. Cho, and S. Lee, “Unsupervised video anomaly detection via normalizing flows with implicit latent features,” Pattern Recognit., vol. 129, p. 108703, Sep. 2022, doi: 10.1016/j.patcog.2022.108703.
[3] L. Wang, H. Tan, F. Zhou, W. Zuo, and P. Sun, “Unsupervised Anomaly Video Detection via a Double-Flow ConvLSTM Variational Autoencoder,” IEEE Access, vol. 10, pp. 44278–44289, 2022, doi: 10.1109/ACCESS.2022.3165977.
[4] X. Wang et al., “Robust Unsupervised Video Anomaly Detection by Multipath Frame Prediction,” IEEE Trans. Neural Networks Learn. Syst., vol. 33, no. 6, pp. 2301–2312, Jun. 2022, doi: 10.1109/TNNLS.2021.3083152.
[5] C. Huang et al., “Self-Supervised Attentive Generative Adversarial Networks for Video Anomaly Detection,” IEEE Trans. Neural Networks Learn. Syst., vol. 34, no. 11, pp. 9389–9403, Nov. 2023, doi: 10.1109/TNNLS.2022.3159538.
[6] R. Leider, “The Modern Common Law of Crime,” J. Crim. Law Criminol., vol. 111, no. 2, pp. 407–499, Jan. 2021. [Online]. Available at: https://scholarlycommons.law.northwestern.edu/jclc/vol111/iss2/2.
[7] T. Abam, “Impact of Terrorism on Society Insecurities,” J. Anthropol. Reports, vol. 5, no. 5, pp. 9–10, Sep. 2022. [Online]. Available at: https://www.walshmedicalmedia.com/open-access/impact-of-terrorism-on-society-insecurities-114620.html.
[8] A. Birze, K. Regehr, and C. Regehr, “Workplace Trauma in a Digital Age: The Impact of Video Evidence of Violent Crime on Criminal Justice Professionals,” J. Interpers. Violence, vol. 38, no. 1–2, pp. 1654–1689, Jan. 2023, doi: 10.1177/08862605221090571.
[9] S. Kim, P. Joshi, P. S. Kalsi, and P. Taheri, “Crime Analysis Through Machine Learning,” in 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Nov. 2018, pp. 415–420, doi: 10.1109/IEMCON.2018.8614828.
[10] N. Carmack, “Benefits of Surveillance Cameras in Public Places,” BOS Security, 2022. [Online]. Available at: https://www.bossecurity.com/2022/12/21/benefits-of-surveillance-cameras-in-public-places/.
[11] I. Insider, “Role of CCTV Cameras: Public, Privacy and Protection,” IFSEC Insider | Security and Fire News and Resources, 2021. [Online]. Available at: https://www.ifsecglobal.com/video-surveillance/role-cctv-cameras-public-privacy-protection/.
[12] W. Sultani, C. Chen, and M. Shah, “Real-World Anomaly Detection in Surveillance Videos,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 6479–6488, doi: 10.1109/CVPR.2018.00678.
[13] T. Zhang et al., “Recent Advances in Video Analytics for Rail Network Surveillance for Security, Trespass and Suicide Prevention—A Survey,” Sensors, vol. 22, no. 12, p. 4324, Jun. 2022, doi: 10.3390/s22124324.
[14] R. Nawaratne, D. Alahakoon, D. De Silva, and X. Yu, “Spatiotemporal Anomaly Detection Using Deep Learning for Real-Time Video Surveillance,” IEEE Trans. Ind. Informatics, vol. 16, no. 1, pp. 393–402, Jan. 2020, doi: 10.1109/TII.2019.2938527.
[15] B. Kiran, D. Thomas, and R. Parakkal, “An Overview of Deep Learning Based Methods for Unsupervised and Semi-Supervised Anomaly Detection in Videos,” J. Imaging, vol. 4, no. 2, p. 36, Feb. 2018, doi: 10.3390/jimaging4020036.
[16] T. Alanazi, K. Babutain, and G. Muhammad, “A Robust and Automated Vision-Based Human Fall Detection System Using 3D Multi-Stream CNNs with an Image Fusion Technique,” Appl. Sci., vol. 13, no. 12, p. 6916, Jun. 2023, doi: 10.3390/app13126916.
[17] A. Lentzas and D. Vrakas, “Non-intrusive human activity recognition and abnormal behavior detection on elderly people: a review,” Artif. Intell. Rev., vol. 53, no. 3, pp. 1975–2021, Mar. 2020, doi: 10.1007/s10462-019-09724-5.
[18] Honghai Liu, Shengyong Chen, and N. Kubota, “Intelligent Video Systems and Analytics: A Survey,” IEEE Trans. Ind. Informatics, vol. 9, no. 3, pp. 1222–1233, Aug. 2013, doi: 10.1109/TII.2013.2255616.
[19] C. Liu, Y. Zhang, Y. Xue, and X. Qian, “AJENet: Adaptive Joints Enhancement Network for Abnormal Behavior Detection in Office Scenario,” IEEE Trans. Circuits Syst. Video Technol., vol. 34, no. 3, pp. 1427–1440, Mar. 2024, doi: 10.1109/TCSVT.2023.3295432.
[20] H.-T. Duong, V.-T. Le, and V. T. Hoang, “Deep Learning-Based Anomaly Detection in Video Surveillance: A Survey,” Sensors, vol. 23, no. 11, p. 5024, May 2023, doi: 10.3390/s23115024.
[21] N. C. Tay, T. Connie, T. S. Ong, A. B. J. Teoh, and P. S. Teh, “A Review of Abnormal Behavior Detection in Activities of Daily Living,” IEEE Access, vol. 11, pp. 5069–5088, 2023, doi: 10.1109/ACCESS.2023.3234974.
[22] S.-H. Cho and H.-B. Kang, “Abnormal behavior detection using hybrid agents in crowded scenes,” Pattern Recognit. Lett., vol. 44, pp. 64–70, Jul. 2014, doi: 10.1016/j.patrec.2013.11.017.
[23] D. Bahdanau, K. H. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., pp. 1–15, 2015, [Online]. Available at: https://arxiv.org/abs/1409.0473.
[24] E. Bermejo Nievas, O. Deniz Suarez, G. Bueno García, and R. Sukthankar, “Violence Detection in Video Using Computer Vision Techniques,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6855 LNCS, no. PART 2, Springer, Berlin, Heidelberg, 2011, pp. 332–339. [Online]. Available at: 10.1007/978-3-642-23678-5_39.
[25] Nievas et al., “Computer Analysis of Images and Patterns,” Academic Torrents, pp. 332-339. 2011. [Online]. Available at: https://academictorrents.com/details/70e0794e2292fc051a13f05ea6f5b6c16f3d3635.
[26] Ryoo, “UT-Interaction Dataset,” Papers With Code, 2020. [Online]. Available at: https://paperswithcode.com/dataset/ut-interaction.
[27] D. N. Papanikolopoulos et al., “Monitoring Human Activity,” University of Minnesota [Online]. Available at: https://mha.cs.umn.edu/.
[28] K. Soomro, “UCF101 - Action Recognition Data Set,” Center for Research in Computer Vision at the University of Central Florida, 2013. https://www.crcv.ucf.edu/data/UCF101.php.
[29] N. C. Tay, C. Tee, T. S. Ong, and P. S. Teh, “Abnormal Behavior Recognition using CNN-LSTM with Attention Mechanism,” in 2019 1st International Conference on Electrical, Control and Instrumentation Engineering (ICECIE), Nov. 2019, pp. 1–5, doi: 10.1109/ICECIE47765.2019.8974824.
[30] T. Lei, Y. Zhang, S. I. Wang, H. Dai, and Y. Artzi, “Simple Recurrent Units for Highly Parallelizable Recurrence,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 4470–4481, doi: 10.18653/v1/D18-1477.
[31] K.-E. Ko and K.-B. Sim, “Deep convolutional framework for abnormal behavior detection in a smart surveillance system,” Eng. Appl. Artif. Intell., vol. 67, pp. 226–234, Jan. 2018, doi: 10.1016/j.engappai.2017.10.001.
[32] D. Arifoglu and A. Bouchachia, “Activity Recognition and Abnormal Behaviour Detection with Recurrent Neural Networks,” Procedia Comput. Sci., vol. 110, pp. 86–93, Jan. 2017, doi: 10.1016/j.procs.2017.06.121.
[33] Y. Fan, G. Wen, D. Li, S. Qiu, and M. D. Levine, “Early event detection based on dynamic images of surveillance videos,” J. Vis. Commun. Image Represent., vol. 51, pp. 70–75, Feb. 2018, doi: 10.1016/j.jvcir.2018.01.002.
[34] S. Sharma, R. Kiros, and R. Salakhutdinov, “Action Recognition using Visual Attention,” in ICLR 2016 - 10th International Conference on Learning Representations, 2016, pp. 1–6, [Online]. Available at: http://arxiv.org/abs/1511.04119.
[35] X. Chen, J. Yu, and Z. Wu, “Temporally Identity-Aware SSD With Attentional LSTM,” IEEE Trans. Cybern., vol. 50, no. 6, pp. 2674–2686, Jun. 2020, doi: 10.1109/TCYB.2019.2894261.
[36] A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, “Large-Scale Video Classification with Convolutional Neural Networks,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2014, pp. 1725–1732, doi: 10.1109/CVPR.2014.223.
[37] L. Zhao, L. Zhu, S. Zhao, and X. Ma, “Sequestration and bioavailability of perfluoroalkyl acids (PFAAs) in soils: Implications for their underestimated risk,” Sci. Total Environ., vol. 572, pp. 169–176, Dec. 2016, doi: 10.1016/j.scitotenv.2016.07.196.
[38] M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu, “Spatial Transformer Networks,” Adv. Neural Inf. Process. Syst., vol. 2015-January, pp. 2017–2025, Jun. 2015. [Online]. Available at: https://arxiv.org/abs/1506.02025v3.
[39] G. F. Elsayed, S. Kornblith, and Q. V. Le, “Saccader: Improving accuracy of hard attention models for vision,” Adv. Neural Inf. Process. Syst., vol. 32, no. NeurIPS, p. 13, 2019, [Online]. Available at: https://arxiv.org/abs/1908.07644.
[40] David, “Not afraid of ‘overfitting,’” 2024. [Online]. Available at: http://nooverfit.com/wp/.
[41] K. Greff, R. K. Srivastava, J. Koutnik, B. R. Steunebrink, and J. Schmidhuber, “LSTM: A Search Space Odyssey,” IEEE Trans. Neural Networks Learn. Syst., vol. 28, no. 10, pp. 2222–2232, Oct. 2017, doi: 10.1109/TNNLS.2016.2582924.
[42] H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre, “HMDB: A large video database for human motion recognition,” in 2011 International Conference on Computer Vision, Nov. 2011, pp. 2556–2563, doi: 10.1109/ICCV.2011.6126543.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0