For my undergraduate thesis, I worked on a deep learning-based system for video anomaly detection. In simple words, the goal was to build a model that can look at surveillance video footage and detect whether something unusual or criminal is happening. Examples of such activities include robbery, assault, fighting, arson, burglary, shooting, and road accidents. The main idea was not to replace human monitoring, but to support it. In many places, CCTV cameras are already installed, but the footage is not always watched in real time. So, if a system can automatically detect suspicious activities, it can help authorities respond faster. Below diagram shows the entire pipeline of our proposed system.
Why I Chose This Topic
Crime and public safety are serious concerns in many countries. Surveillance cameras are common in streets, shops, offices, parking lots, and public areas. However, simply recording video is not always enough. A camera can capture an incident, but someone still needs to watch the footage and understand what is happening. This is where artificial intelligence can help. A deep learning model can analyze videos and identify abnormal activities more quickly than manual observation in many cases.
The Main Problem
Video anomaly detection is difficult for several reasons.
First, videos are usually long. A crime or unusual activity may happen only for a few seconds inside a long video. Second, anomalies are rare. Most of the video may contain normal activities, while only a small part contains the actual incident. Third, the meaning of an activity depends on context. For example, running in a playground is normal, but running during a robbery may be suspicious.
So, our challenge was to design a model that could understand both the visual information and the time-based pattern of activities in a video.
Dataset Used
For this project, we used the UCF Crime dataset. It is a public dataset containing real-world surveillance videos of normal and abnormal activities. We did not directly use the full-length videos. Instead, we divided long videos into smaller segments so the model could focus better on the actual activity.
How Our System Works
Our proposed system has Three main parts. A diagram of the system is shown below.
1. Video Segmentation
The original videos were long, and the abnormal event did not always appear throughout the whole video. So, we divided the videos into smaller fixed-length clips. This helped the model learn from the most important parts of the video. See the first part of the diagram for how we segmented the videos. We also manually labeled the segmented clips so the model could learn which type of activity each clip represented.
2. Feature Extraction Using ResNet50
A video contains many frames, and each frame contains a lot of visual information. To extract useful visual features, we used a customized ResNet50 model. ResNet50 helped us convert video frames into meaningful feature embeddings. See the second part of the diagram for how we extracted features from video frames. The diagram below shows the ResNet50 architecture we used for feature extraction.
Instead of using only the final layer, we combined features from different levels of the network. This helped the model capture both simple visual details and more complex patterns.
3. Temporal Learning Using SRU++
After extracting visual features from each video frame, we still needed the model to understand how those features changed over time. This is important because an anomaly is not always visible from a single frame. For example, fighting, robbery, or an accident usually becomes clear only when we observe a sequence of actions.
For this part, we used SRU++, which stands for Simple Recurrent Unit Plus Plus. SRU++ was mainly designed for sequential language processing tasks, such as language modeling. In language processing, a model reads words or tokens one after another and tries to understand the meaning based on their order. We used the same idea for videos.
Instead of words, our sequence contained frame embeddings. Each embedding represented the visual information of a video frame. So, the SRU++ model looked at these frame embeddings one by one and learned how the activity changed across time.
The reason we chose SRU++ is that it is designed to be faster and more efficient than many traditional recurrent models like LSTM. We also added an attention mechanism so the model could focus more on the important parts of the video sequence. This was useful for our task because the actual anomaly may appear only in a few frames of a video segment.
In simple terms, ResNet50 helped us understand what was visible in the video, while SRU++ helped us understand what was happening over time.
Results
Our final model was called ResSRUppCW. It achieved 96.26% AUROC for binary anomaly detection and 93.72% accuracy for multiclass classification.
Binary anomaly detection means identifying whether a video is normal or abnormal. Multiclass classification means identifying the specific type of anomaly, such as robbery, fighting, arson, or shooting. The results showed that our model performed better than several existing methods tested on the UCF-Crime dataset.
What I Learned
This thesis helped me understand that building an AI system is not only about choosing a model. Data preparation, preprocessing, labeling, feature extraction, and evaluation are equally important. I also learned that real-world video data is messy. Some videos are blurry, some incidents happen far from the camera, and some activities look similar to normal behavior.
These issues make video anomaly detection a challenging but meaningful research problem.
Limitations
Our model performed well, but it was not perfect. Some classes had fewer samples, which affected performance. Also, videos with poor lighting, distant camera views, crowded scenes, or unclear actions could still confuse the model. Another important limitation is ethics. Surveillance-based AI systems must be used responsibly. Privacy, consent, bias, and misuse should always be considered before real-world deployment.
Future Work
In the future, this work can be improved by using more diverse datasets, adding explainability features, and testing the model in real-time environments. Another interesting direction is criminal activity prediction, where the goal would be to detect early signs of suspicious activity before the incident fully happens.
Final Thoughts
My thesis focused on using deep learning to detect abnormal activities in surveillance videos. The project combined video segmentation, ResNet50-based feature extraction, SRU++ temporal learning, and attention mechanisms. The final result was a model that showed strong performance on the UCF-Crime dataset and demonstrated the potential of AI-assisted surveillance. This project was a valuable learning experience and gave me a deeper understanding of computer vision, deep learning, and real-world AI challenges.