EyeNet: Turning a Webcam Feed Into Actionable Security Incidents

I built a real-time campus surveillance system that turns raw video into actionable incidents — here's how EyeNet works

Most "AI surveillance" demos I see online stop at object detection: a bounding box, a confidence score, done. That's not actually useful for a security team. A camera that just says "knife detected" a hundred times a second because of one flickering frame is worse than no camera at all.

So I built EyeNet — a camera-first, real-time incident response system that treats detection as just the first step in a pipeline that ends in a structured, queryable, actionable event.

The problem

Talk to any campus security or discipline team and you'll hear the same pain points:

• Humans can't watch 20 camera feeds at once, all day, every day.
• Alerts happen over phone calls or WhatsApp — no structure, no traceability, no history.
• Naive CV systems throw false positives constantly unless you enforce persistence and cooldowns.
• Nobody has a reviewable audit trail with snapshots when something does go wrong.

EyeNet's goal: turn a live video feed into structured incidents — not noise.

Architecture

Camera → SharedFrameBuffer → DetectionPipeline → ObjectTracker → DetectionEvent
                                                                       ↓
                                                              EventBus (priority queue)
                                                            ↙        ↓        ↘
                                                     DB Handler   Notifications   SSE → Dashboard

A few decisions I'd make again:

One camera reader, many consumers. SharedFrameBuffer owns /dev/video0 exclusively so the pipeline, dashboard stream, and any future consumer aren't fighting over the same device.

Multi-signal detection per frame, not just one model.

• Face recognition against enrolled encodings (face-recognition / dlib), matched via L2 distance against a threshold.
• Uniform compliance — for known faces only, I estimate a torso ROI below the face box, convert to HSV, and check for light-blue fabric ratio. No fancy classifier needed for this one.
• Hazard detection via YOLOv8, filtered through a keyword allow-list (knife, gun, fire, smoke, axe, sword...) with per-class confidence thresholds and a minimum bounding-box area to kill tiny false positives.

Tracking before alerting. Every detection goes through ByteTrack (via supervision, with a fallback ID scheme if it's not installed). A hazard or unknown face only becomes an alert once its track has persisted for a minimum number of frames — and only once per track. This single decision killed the vast majority of my false-positive alerts.

Severity-aware event bus. Events aren't handled FIFO — they go into a priority queue where CRITICAL (gun, fire, bomb) always jumps ahead of LOW (uniform violations). An in-process EventBus with multiple worker threads dispatches to registered handlers (DB, notifications, SSE) independently, so a slow SMTP call never blocks persistence.

Anomaly scoring, not just a label. Every event gets a 0–100 urgency score from a base weight per detection type, the model's confidence, and how long the track has persisted. This score gets stored in the alert metadata and gates SMS notifications (only fires above a threshold).

Cooldowns at two levels. A system-wide cooldown keyed off severity (1 min for CRITICAL up to 24 hours for LOW) plus a separate in-memory cooldown inside the SMS sender keyed by event subtype. Belt and suspenders against alert spam.

Persistence & the dashboard

Alerts and metrics land in SQLite with WAL mode enabled — cheap way to get decent concurrent-read behavior while the pipeline keeps writing. The Flask dashboard then gives you:

• Live MJPEG video feed
• Live alerts over Server-Sent Events (no websockets needed)
• Alert filtering by type/severity, snapshot thumbnails, and an acknowledge button
• Hourly analytics aggregated over the last 24 hours
• A metrics panel polling fps, detection counts, and processing latency every few seconds

Auth is deliberately simple for now — single bcrypt-hashed admin account, Flask session cookies. Multi-role permissions is on the roadmap, not solved yet.

Notifications

• SMS via Twilio for unknown faces and hazards, gated by anomaly score ≥ 25
• Email via SMTP for uniform violations, addressed to the student's derived institutional email

Stack

Python 3.9, Flask, OpenCV, Ultralytics YOLOv8, face-recognition (dlib), supervision for ByteTrack, SQLite (WAL), Twilio, bcrypt, Docker Compose with camera device passthrough.

What I'd tackle next

• Swap the in-process SSE queue for something multi-process safe (Redis pub/sub) if this needs to scale past a single process
• Proper RBAC instead of one admin account
• Bind face encodings to the DB instead of a standalone pickle file
• Rate limiting on the API routes
• A real incident review workflow — notes, assignment, escalation states — instead of just acknowledge/dismiss

Demo

Here's a walkthrough of it running: youtube.com/watch?v=iarosznhHE8

Try it / poke at the code

Repo: github.com/SplinterSword/EyeNet

If you're working on anything similar — event-driven CV pipelines, alert fatigue problems, or SQLite-under-concurrent-load setups — I'd love to compare notes. Happy to answer questions about any of the design decisions above in the comments.