far-sight: I Built My Friend a Local AI That Catches Their Cat on the Kitchen Counter

Milan Lazarevic — Sun, 04 Oct 2026 15:29:16 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

My friend has a cat who believes the kitchen counter is theirs. Every evening my friend comes home to paw prints by the sink and the occasional knocked-over glass, with no idea when it happened or how often. They didn't want a cloud pet cam streaming their apartment to someone else's server, and they didn't want to scrub through hours of footage either.

So I built far-sight, a small Linux program that watches a webcam and keeps asking a local vision-language model one plain-English question: "Does this image show a cat on the kitchen counter?" When the answer is yes, it saves a photo of that exact moment, with the capture time in the filename and stamped onto the image.

far-sight -m SmolVLM-500M-Instruct-Q8_0.gguf \
  --mmproj mmproj-SmolVLM-500M-Instruct-Q8_0.gguf \
  -p "a cat on the kitchen counter"

Once it's set up in their kitchen, my friend will come home to a folder of timestamped photos and an events.csv log that shows when the cat was up there, how often, and the evidence.

Not just for cats

Nothing in far-sight is cat-specific. The event is just a sentence passed to -p, so the same binary covers anything you can describe and point a camera at:

Other pets: "a dog lying on the couch", "a cat scratching the sofa". Find out what happens while you're out.
Front door and driveway: "a package on the doorstep", "a car parked in the driveway". Get a timestamped record of deliveries and arrivals.
Garden and wildlife: "a bird at the feeder", "a deer in the garden". A nature log without watching hours of footage.
Workshop: "a tangled mess of plastic on the 3D printer bed". Catch a failed print before it wastes the whole spool.
Hands-free photos: "a person waving at the camera". Wave and it takes the picture.
Around the house: "the garage door is open", "someone sitting at my desk". Know when something changed, and when.

The --on-match hook turns a detection into an action. It can send a desktop notification, push the photo to a phone, publish to MQTT for Home Assistant, or run any script you like, so far-sight works as a general "tell me when X happens" building block. Swapping SmolVLM for a bigger open model such as Qwen2.5-VL or Gemma 3 is just a different file, for when a prompt needs more visual understanding than a 500M model has.

Demo

I built far-sight over the challenge weekend and haven't had the chance to set it up at my friend's place yet, so I don't have footage of the actual culprit on the actual counter. Instead, I tested it with a stock clip of a cat on a kitchen counter, played into a virtual webcam (v4l2loopback + ffmpeg). far-sight can't tell the difference: it opens /dev/video10 exactly like a real camera, so this is the same binary and the same pipeline that will run in my friend's kitchen, just with a different video source. The clip starts on an empty counter, then the cat shows up, walks along the counter and leaves.

far-sight -m SmolVLM-500M-Instruct-Q8_0.gguf \
  --mmproj mmproj-SmolVLM-500M-Instruct-Q8_0.gguf \
  -d /dev/video10 -p "a cat on the kitchen counter" --cooldown 15s -v \
  --on-match 'notify-send "far-sight" "Cat on the counter at $FARSIGHT_TIME"'

Here's the log from one pass of the clip, running on a laptop CPU:

[2026-10-04 17:06:37.645] camera /dev/video10 opened at 1280x720 YUYV
[2026-10-04 17:06:37.785] watching for "a cat on the kitchen counter" every 1000ms (saving to captures)
[2026-10-04 17:06:37.816] p_yes=0.002 no streak=0 (796 ms)
[2026-10-04 17:06:38.819] static scene (diff 0.29), skipped
[2026-10-04 17:06:39.821] static scene (diff 0.41), skipped
[2026-10-04 17:06:40.785] p_yes=0.987 MATCH streak=1 (786 ms)
[2026-10-04 17:06:41.786] p_yes=0.998 MATCH streak=2 (722 ms)
2026-10-04 17:06:41.786 captures/2026-10-04_17-06-41.786.jpg 0.998
[2026-10-04 17:06:42.787] p_yes=0.997 MATCH streak=1 (719 ms)
...
[2026-10-04 17:06:50.795] p_yes=0.989 MATCH streak=7 (749 ms)
[2026-10-04 17:06:51.796] p_yes=0.002 no streak=0 (765 ms)
[2026-10-04 17:06:52.800] static scene (diff 0.33), skipped

The empty counter scores 0.002, and once nothing is moving the model isn't run at all. The cat arrives, two matches in a row fire the event, and the photo is saved within about 0.7 s of each check. The cat keeps posing for another nine seconds, but the cooldown means my friend will get one photo per visit, not ten.

The saved photo has the capture time burned into the corner:

And events.csv keeps the running record of every visit. Here are both passes of the clip:

captured_at,image,score,prompt
2026-10-04T17:06:41.786+02:00,"2026-10-04_17-06-41.786.jpg",0.998,"a cat on the kitchen counter"
2026-10-04T17:07:01.807+02:00,"2026-10-04_17-07-01.807.jpg",0.998,"a cat on the kitchen counter"

Try it yourself. The whole demo is one script in the repo, examples/cat-on-counter. It downloads the model and the clip, sets up the video and runs far-sight:

sudo modprobe v4l2loopback video_nr=10 card_label="far-sight demo" exclusive_caps=1
./examples/cat-on-counter/run-demo.sh

Demo footage: "Ginger Cat Walks on Sunny Kitchen Counter" by Alex Moliski from Pexels, used under the Pexels License.

Code

MrLaki5 / far-sight

far-sight

Watches a webcam and asks a small local vision-language model whether something you describe is happening, such as "a person waving" or "the garage door is open". When it is, far-sight saves a JPEG of that moment with the capture time in the filename and burned into the image, and appends a row to events.csv.

It is a single C++ binary for Linux. It links llama.cpp (including its libmtmd multimodal library) statically and captures from the camera through V4L2 directly, so it needs no OpenCV or other runtime dependencies.

Frame from the cat-on-counter example. Footage: "Ginger Cat Walks on Sunny Kitchen Counter" by Alex Moliski from Pexels.

Quick start

cmake -S . -B build && cmake --build build -j       # build
./scripts/download-model.sh                         # get SmolVLM-500M (~540 MB)
./build/far-sight -m models/SmolVLM-500M-Instruct-Q8_0.gguf \
    --mmproj models/mmproj-SmolVLM-500M-Instruct-Q8_0.gguf \
    -p "a cat on the kitchen counter"               # watch /dev/video0

Photos…

View on GitHub

The README has a three-command quick start for a real webcam, and examples/cat-on-counter reproduces the demo above without one. The project is Apache-2.0.

How I Built It

The AI at the core is entirely open:

Model: SmolVLM-500M-Instruct, an open-weight vision-language model (Apache 2.0), in the Q8_0 GGUF build from ggml-org. The model and its vision projector come to about 540 MB, small enough to run on a laptop CPU.
Runtime: llama.cpp and its multimodal library libmtmd, statically linked into a single C++ binary. There's no Python, no server and no OpenCV. CMake fetches llama.cpp at a pinned release tag; the build is CPU by default, and -DGGML_CUDA=ON or -DGGML_VULKAN=ON turns on GPU support.

Each check goes through five steps:

Grab the freshest frame. far-sight reads the camera through V4L2 directly and decodes MJPEG with stb_image. Frames that piled up while the model was busy are thrown away, and the time comes from the driver's buffer timestamp, so the saved time is when the photo was taken, not when the model finished.
Skip boring frames. A 64×48 grayscale thumbnail is compared with the last frame the model saw. If almost nothing changed, the model doesn't run, so a quiet afternoon in an empty kitchen costs almost no compute.
Ask the model, but read its mind instead of its words. Instead of generating an answer and parsing text, far-sight reads the model's logits for the first answer token and computes P(yes) / (P(yes) + P(no)). That takes one forward pass and gives a confidence score between 0 and 1 that I can put a threshold on.
Debounce. It fires only after N matches in a row and then waits out a cooldown, so one visit to the counter means one photo, not thirty.
Save the moment. The JPEG gets the capture time burned in with a tiny built-in 5×7 pixel font, a row goes into events.csv, and an optional --on-match shell hook gets the image path and time as environment variables. That hook can show a desktop notification or push the photo to a phone.

On a laptop CPU each check takes about 0.7–0.8 s at 512 px, fast enough to check every second. It runs as a systemd user service, so it starts on login and stays out of the way.

Why Does Open Innovation Matter?

The camera stays in the home. This is a webcam pointed at someone's kitchen. With open weights and local inference, no frame ever leaves the machine: there's no account, no upload and no privacy policy to trust.
It's free to run all day. Checking once a second is 86,400 checks a day. With a hosted vision API that becomes a monthly bill; locally it's a bit of electricity, and the motion gate skips most of the checks anyway.
I could get inside the model. The yes/no confidence score comes from reading the raw logits for the "yes" and "no" tokens. That only works because I run the model myself; hosted APIs usually give back text, not the model's full output distribution.
It keeps working. There's no rate limit, no API deprecation and no outage on someone else's side. When the Wi-Fi drops, the cat still gets caught. A better open model later, such as Qwen2.5-VL or Gemma 3, is a file swap, not a rewrite.
It's open all the way down. llama.cpp is MIT, SmolVLM is Apache 2.0, stb is public domain, and far-sight itself is Apache 2.0.

My Agent Session

I built far-sight in a single session pairing with Claude Code as my coding agent. It read llama.cpp's current libmtmd headers so the code targets the real API, wrote the C++ and CMake, built it, and tested it against my real webcam, then against a cat clip played through a virtual camera to produce the demo above. The AI inside the product is fully open-source; the coding agent was the one closed piece of the workflow.

DEV Community: Milan Lazarevic