Deep Context-Aware Feature Extraction for Anomaly Detection in Surveillance Videos
Received: 2 December 2024 | Revised: 29 December 2024 | Accepted: 7 January 2025 | Online: 3 April 2025
Corresponding author: Anuja Phapale
Abstract
Surveillance video analysis plays a crucial role in ensuring public safety and security. Developing a context-aware framework for anomaly detection in surveillance videos is motivated by the need for enhanced security, safety, and efficiency in various domains. Context-aware anomaly detection depends on spatiotemporal features that help the model understand the context of anomalies in surveillance videos. This study aimed to provide a novel deep learning-based context-aware approach to feature extraction to detect anomalies in surveillance videos. The proposed method integrates ResNet50 for spatial feature extraction and 3D Convolutional Neural Network (CNN) for temporal feature extraction. This method identifies six anomalous activities, namely abuse, arrest, fighting, robbery, shooting, and road accidents, using the UCF-Crime dataset. The proposed integrated ResNet50 and 3D CNN model achieves promising accuracy for the six classes, such as 95% for abuse, 93% for arrest, 95.22% for fighting, 94.44% for robbery, 93% for shooting, and 94.22% for road accidents. By combining spatiotemporal features, the proposed model detects anomalies in behavior and unexpected movements, which makes it useful for security monitoring where deviations from normal behavior indicate anomalous events. This research contributes to advancing the capabilities of surveillance systems, enhancing public safety, and enabling proactive security measures in diverse urban environments.
Keywords:
deep learning, ResNet50, surveillance video analysis, spatiotemporal analysis, UCF-crime dataset, 3D CNNDownloads
References
S. W. Khan et al., "Anomaly Detection in Traffic Surveillance Videos Using Deep Learning," Sensors, vol. 22, no. 17, Aug. 2022, Art. no. 6563.
K. P. Sanal Kumar and R. Bhavani, "Human activity recognition in egocentric video using HOG, GiST and color features," Multimedia Tools and Applications, vol. 79, no. 5–6, pp. 3543–3559, Feb. 2020.
N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 2005, vol. 1, pp. 886–893.
B. Sabzalian, H. Marvi, and A. Ahmadyfard, "Deep and Sparse features For Anomaly Detection and Localization in video," in 2019 4th International Conference on Pattern Recognition and Image Analysis (IPRIA), Tehran, Iran, Mar. 2019, pp. 173–178.
A. Temizel, P. Güler, and T. T. Temizel, "Real-Time Global Anomaly Detection for Crowd Video Surveillance Using SIFT," in 5th International Conference on Imaging for Crime Detection and Prevention (ICDP 2013), London, UK, 2013.
K. Seemanthini, S. S. Manjunath, G. Srinivasa, B. Kiran, and P. Sowmyasree, "A Cognitive Semantic-Based Approach for Human Event Detection in Videos," in Smart Trends in Computing and Communications, 2020, pp. 243–253.
N. Dalal, B. Triggs, and C. Schmid, "Human Detection Using Oriented Histograms of Flow and Appearance," in Computer Vision – ECCV 2006, 2006, pp. 428–441.
J. Kim and K. Grauman, "Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates," in 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, Jun. 2009, pp. 2921–2928.
M. Qasim and E. Verdu, "Video anomaly detection system using deep convolutional and recurrent models," Results in Engineering, vol. 18, Jun. 2023, Art. no. 101026.
Z. K. Abbas and A. A. Al-Ani, "An adaptive algorithm based on principal component analysis-deep learning for anomalous events detection," Indonesian Journal of Electrical Engineering and Computer Science, vol. 29, no. 1, Jan. 2022, Art. no. 421.
S. Dubey, A. Boragule, J. Gwak, and M. Jeon, "Anomalous Event Recognition in Videos Based on Joint Learning of Motion and Appearance with Multiple Ranking Measures," Applied Sciences, vol. 11, no. 3, Feb. 2021, Art. no. 1344.
S. Ul Amin et al., "EADN: An Efficient Deep Learning Model for Anomaly Detection in Videos," Mathematics, vol. 10, no. 9, May 2022, Art. no. 1555.
W. Ullah, A. Ullah, T. Hussain, Z. A. Khan, and S. W. Baik, "An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos," Sensors, vol. 21, no. 8, Apr. 2021, Art. no. 2811.
N. Gupta and B. B. Agarwal, "Suspicious Activity Classification in Classrooms using Deep Learning," Engineering, Technology & Applied Science Research, vol. 13, no. 6, pp. 12226–12230, Dec. 2023.
W. Ullah, T. Hussain, F. U. M. Ullah, M. Y. Lee, and S. W. Baik, "TransCNN: Hybrid CNN and transformer mechanism for surveillance anomaly detection," Engineering Applications of Artificial Intelligence, vol. 123, Aug. 2023, Art. no. 106173.
Y. Lee and P. Kang, "AnoViT: Unsupervised Anomaly Detection and Localization With Vision Transformer-Based Encoder-Decoder," IEEE Access, vol. 10, pp. 46717–46724, 2022.
Q. Zhang, H. Wei, J. Chen, X. Du, and J. Yu, "Video Anomaly Detection Based on Attention Mechanism," Symmetry, vol. 15, no. 2, Feb. 2023, Art. no. 528.
C. Wu, S. Shao, C. Tunc, P. Satam, and S. Hariri, "An explainable and efficient deep learning framework for video anomaly detection," Cluster Computing, vol. 25, no. 4, pp. 2715–2737, Aug. 2022.
B. Wang, C. Yang, and Y. Chen, "Detection Anomaly in Video Based on Deep Support Vector Data Description," Computational Intelligence and Neuroscience, vol. 2022, pp. 1–6, May 2022.
V. Mahadevan, W. Li, V. Bhalodia, and N. Vasconcelos, "Anomaly detection in crowded scenes," in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, Jun. 2010, pp. 1975–1981.
W. Li, V. Mahadevan, and N. Vasconcelos, "Anomaly Detection and Localization in Crowded Scenes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 1, pp. 18–32, Jan. 2014.
"Anomaly Detection and Localization in Crowded Scenes." [Online]. Available: http://www.svcl.ucsd.edu/projects/anomaly/.
"Avenue Dataset." [Online]. Available: https://www.cse.cuhk.edu.hk/leojia/projects/detectabnormal/dataset.html.
C. Lu, J. Shi, and J. Jia, "Abnormal Event Detection at 150 FPS in MATLAB," in 2013 IEEE International Conference on Computer Vision, Sydney, Australia, Dec. 2013, pp. 2720–2727.
W. Sultani, C. Chen, and M. Shah, "Real-World Anomaly Detection in Surveillance Videos," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, Jun. 2018, pp. 6479–6488.
"UCF-Crime-database",[Online] Available: https://www.dropbox.com/sh/75v5ehq4cdg5g5g/AABvnJSwZI7zXb8_myBA0CLHa?dl=0
W. Luo, W. Liu, and S. Gao, "A Revisit of Sparse Coding Based Anomaly Detection in Stacked RNN Framework," in 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, Oct. 2017, pp. 341–349.
"Monitoring Human Activity - Detection of Events." [Online]. Available: https://mha.cs.umn.edu/proj_events.shtml.
Downloads
How to Cite
License
Copyright (c) 2025 Anuja Phapale, Sukhada Bhingarkar
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
