One camera. One NVIDIA Jetson Orin Nano Super. Every model runs on the board.
That was the brief for Sentinel, our final-year project at Dayananda Sagar University: spot people, follow them across the frame, and open an incident when someone steps into a zone you care about. No cloud AI anywhere in the loop.
We expected the hard part to be the models. It wasn't.
See it first: the project website has a hardware tour and a demo dashboard you can click around in. It is the real dashboard code replaying a scripted scene, so no Jetson is needed.
The dashboard with synthetic demo data. No camera or model was running.
The pipeline
| Step | What does it |
|---|---|
| See | YOLOv8n on TensorRT detects people |
| Follow | ByteTrack keeps each person's track |
| Decide | Zone rules open incidents in a local SQLite database |
| Tell | A restart-safe outbox delivers alerts (Telegram in our tests) |
| Show | A dashboard that listens on 127.0.0.1 only |
Two optional lanes sit beside it: face recognition (YuNet + Facenet512) and a local vision-language model (LFM3-VL-2B through llama.cpp). Both are still under validation, so they are off by default. More on that below.
Lesson 1: your slowest model shouldn't make your fastest decisions
On a small board, models finish at very different speeds. The detector keeps up with the stream. Face matching is slower. A vision-language model is slower still.
Now imagine the camera drops and reconnects. A slow model is still chewing on a frame from before the reconnect. When its answer finally lands, the naive pipeline treats it as news about now.
That is how an old frame fires a new alert.
Our rule: every observation is evidence about one specific frame, never about "now". So every observation carries three stamps:
- the frame's source time
- the boot ID of the run that produced it
- a stream epoch that changes on every reconnect
Before any result can drive a decision, it has to pass a check like this (a simplified sketch, not the project's exact code):
def can_drive_live_decision(obs, now, current):
if obs.boot_id != current.boot_id:
return False # from a previous run
if obs.stream_epoch != current.stream_epoch:
return False # from before a reconnect
if now - obs.frame_time > MAX_AGE:
return False # simply too old
return True
A result that fails is not thrown away. It can still annotate history. It just never gets to act on the present.
One more detail that mattered: age is measured with an injected monotonic clock, not wall time. That makes the rule testable. A fake clock lets a test replay "the model answered late" without waiting for a real model to be slow.
Lesson 2: "I don't know" is a valid answer
A monitoring system that guesses is worse than one that stays quiet. Three rules came out of that:
- A face is tied to at most one tracked person. If ownership is ambiguous, it stays unresolved.
- An unknown face is never labelled a stranger. Not recognising someone is not evidence about who they are.
- Fire or smoke reported by the scene model alone is never a critical alarm.
The dashboard follows the same idea. Empty, fresh, stale and offline are four different states, and it shows each one explicitly instead of leaving an old picture on screen.
Lesson 3: an alert that dies with the process was never an alert
An in-memory queue loses everything on a restart. So incidents and their pending alerts are written to SQLite, and a worker delivers from that outbox with retries. If Sentinel restarts between "incident opened" and "alert sent", the alert still goes out afterwards.
Lesson 4: test without the hardware
Most of Sentinel can be tested on a laptop. The portable suite needs no camera, GPU or model, and CI runs it on every push under Python 3.10 and 3.12. On the commit the beta was built from, 4,224 tests passed.
The trick is keeping hardware adapters at the edges, so importing the core never requires CUDA or TensorRT.
What isn't done
Sentinel is an experimental beta, and we would rather say so than oversell it:
- Face recognition has not passed its formal validation.
- Chat with the local vision-language model is not admitted for normal use.
- The install package has run only on our own Jetson.
The README keeps a feature table that says what is admitted and what is not, with the evidence behind each claim.
Try it
- In a browser: the project website and its demo dashboard
- On any computer with Python 3.10+: clone the repo and run the test suite
- On a Jetson Orin Nano: the experimental install package in the repo
Code (MIT): https://github.com/hemkesh2021-dotcom/Sentinel_Surveillance
If you run a slow model next to a fast one, how do you keep the slow one's answers honest? I'd like to hear how others handle it.

Top comments (0)