The modern age of video surveillance is evolving beyond the conventional camera-to-recorder-to-review paradigm.
Real-time applications require that the system be capable of recognizing any event, processing the information instantly, and providing relevant information to the operator. Such a system requires an effective blend of edge computing, AI inference, cloud computing, and real-time communications.
A Simple Architecture
IP Cameras
↓
Edge Gateway
↓
AI Detection
↓
Event Processing
↓
Cloud VMS
↓
Operator
↓
Response
Each layer has a specific responsibility.
1. Edge Gateway
Edge Gateway accepts camera streams via RTSP and is capable of performing functions like:
- Stream management
- Local buffering
- Device health monitoring
- Selected AI processing
Processing part of the data closer to the camera can help save bandwidth and improve the performance.
2. Turning Video Into Events
Rather than streaming all the video frames through the entire application process, computer vision can generate events in structured format:
For instance:
{
"camera": "camera-01",
"object": "person",
"zone": "restricted",
"confidence": 0.93
}
This event is much smaller than the original video stream and can be processed separately.
Event processing pipeline is normally:
Camera -> AI Detection -> Event -> Rules Engine -> Operator
3. Rules Add Context
Detection by an AI does not necessarily mean a security breach exists.
A rules engine can consider further details such as:
- Time
- Location
- Detection zone
- Confidence level
- Site status
For example:
Person detected + Restricted zone + After hours = High priority
This method can be used to decrease false alerts.
4. Human Verification
While AI may be helpful in identifying and prioritizing activity, human validation can give further context.
AI Detection -> Rules Engine -> Operator -> Verification -> Response
In this case, the system takes care of the routine tasks and the operator deals with events requiring context.
5. Low-Latency Video
To enable investigations into an event, the system will need to have access to live video with low latency.
WebRTC technology can be used for low-latency video delivery to browsers, and WebSockets technology for delivering event notifications to operator interfaces.
Camera -> Edge Gateway -> Media Gateway -> WebRTC -> Browser
Live-video delivery and archiving of videos could be decoupled to enable optimization of pipelines for their own respective purposes.
6. Design for Failure
In practice, real-world networks are imperfect.
An edge system needs to account for:
- Local buffering
- Automatic reconnection
- Device health monitoring
- Network failover
- Store-and-forward events
A device being offline should be handled differently than a device sending events about lack of activity.
For instance:
Camera Offline -> Health Event -> Administrator
It would help ensure that a network connection problem is not confused with a quiet environment.
7. The Complete Architecture
The architecture described above separates processing of large volumes of video from small event processing while maintaining the whole process chain intact.
Final Thoughts
A contemporary AI-based video surveillance system is a distributed real-time system that follows:
DETECT -> VERIFY -> DETER -> RESPOND
The key architectural idea is to treat video as a real-time data source, not simply something to record.
The fundamental architectural principle is to consider video as a real-time stream rather than something to be recorded.
Time-critical processing can be done at the edge, cloud services can be used for coordination, and human operators can bring context into the picture.
The end product is a flexible basis for developing modern AI-enabled video surveillance systems.
Top comments (0)