I built an AI field guide that works with the internet off, live on stream

Freddy Haug — Mon, 09 Nov 2026 10:40:40 +0000

This is a submission for the Hacktoberfest 2026 DEV Challenge, Week 3: Slow and Steady.

What I Built

Field Kit is an offline reference for the outdoors. Your PC is the box: it runs a small open model and a search index over about 19,000 Simple English Wikipedia articles. Any phone on the same Wi-Fi opens the app in its browser. After a one-time download, nothing leaves the machine.

It has three tools:

Field Notes: ask a question, get a streamed answer with numbered sources from the offline library.
Reference: semantic search and an article reader for the whole pack.
Survey: take a photo of a plant, bug or mushroom. A small vision model guesses what it is, then looks the guess up in the same library so you can check it.

It's for the moments your usual AI tools can't help: a trail with no signal, a cabin, a power or network outage, or anyone who would rather their questions didn't leave their house.

That's the "slow and steady" part. It isn't the fastest or smartest AI you can use. It's the one that's still there when everything else is down, quietly doing the looking-up for you.

I built it live in a two-hour MLH stream for Hacktoberfest on October 19.

Demo

Home	Field Notes	Reference	Survey

Code

Haugf / offline-knowledge-box

Field Kit — Offline Knowledge Box

An open-source, fully offline question-answering box, built live on stream for Hacktoberfest 2026 A small open-weight model answers from a local knowledge pack built from Simple English Wikipedia. It runs natively on Windows (no Docker, no WSL, no Ollama): Python + transformers on the GPU, with a phone-first web UI in the Field Kit style (US-map palette: map paper public-land mint, forest ink, road red for danger only).

Your PC is the box: it runs the models and the search. Any phone on the same Wi‑Fi opens the UI in its browser. Running the model on the phone itself is a goal for a future stream.

Home	Field Notes	Reference	Survey

Status, ask box, tools	Answers with sources	Offline Wikipedia search	Identify by photo

How it works

flowchart LR
    subgraph Phone["Phone browser"]
        UI["Field Kit web UI"]
    end
    subgraph PC["Your PC: the box"]
        API["server.py<br/>FastAPI"]
        RAG["Retrieval<br/>MiniLM

…

View on GitHub

One command on Windows:

.\run.ps1

On first run it sets up a Python environment, builds the knowledge pack (a few minutes on a GPU), and prints the address to open from your phone.

How I Built It

The stack is deliberately small: FastAPI, Hugging Face transformers on the GPU, sentence-transformers for search, and one HTML file for the UI. Four files do the work: server.py, ingest.py, run.ps1 and static/index.html.

The knowledge pack. ingest.py streams Simple English Wikipedia (wikimedia/wikipedia, 20231101.simple) from the Hugging Face Hub. It splits 20,000 articles into about 60,000 chunks and embeds them with all-MiniLM-L6-v2. The result is two files on disk: the vectors and the chunks.

Answering. A question gets embedded and matched against the chunks. The best four go to the chat model, which streams its answer back to the phone with [1]-style citations. Every answer links to the articles it used.

Survey. The phone shrinks the photo to 1024 px before sending it. On the PC, SmolVLM2 2.2B answers in a fixed NAME / SCIENTIFIC / FEATURES format, and that name is searched in the library. A title match is marked EXACT MATCH; everything else is RELATED.

One model on the GPU at a time. My stream PC has an RTX 3070 with 8 GB. Rather than fight for memory, the server keeps either the chat model or the vision model loaded and swaps when you switch tools.

Things that went sideways (and stayed in)

I dropped my own plan. I'd prepped a Docker + Ollama setup with GPU, laptop and Pi-like tiers. On the day, running Python natively on Windows with transformers was simpler and more direct, so the Docker stack came out of the repo during the build. Planning helped anyway: I knew exactly what the end state should be.

A library bug. SmolVLM2 failed on every image under transformers 5.x. The model groups image tokens by its vision patch size (14) instead of tokens per image tile (81). server.py patches inputs_merger until it's fixed upstream. Debugging that on stream was the best ten minutes of the session.

The vision model is confidently wrong sometimes. In testing, a Christmas cactus came back as "rose" with full confidence. That's why every Survey result carries a warning ("An educated guess, not a fact. Never eat or handle anything because an app named it.") and links the library article so you can compare for yourself.

Performance

Measured on the stream PC (RTX 3070, 8 GB) on October 19:

Measure	SmolLM2 135M	SmolLM2 1.7B	SmolVLM2 2.2B (Survey)
Speed (median of 5 questions)	30.1 tok/s	45.0 tok/s	—
Time to first words	2.3 s	2.4 s	—
Full answer	4.7 s	7.2 s	—
First photo (loads the model)	—	—	16.6 s
Each photo after (median of 4)	—	—	1.6 s

Yes, the 135M model is slower per token than the 1.7B. It's so small that Python overhead on each token, not the GPU, sets the pace. Bigger isn't always slower.

Why Does Open Innovation Matter?

For an offline tool, open isn't a nice-to-have. It's the whole thing working.

A closed API can't answer when the network is down. A model you can download can, but how open it is still matters, so part of the stream was a segment called "How open is open?". It walked through the range from closed APIs to open weights under a restrictive license, open weights under a permissive license, and fully open models with published weights, training data and training code.

Field Kit defaults to the fully open end:

Model	Used for	Openness
SmolLM2 135M / 1.7B	Chat (default 1.7B)	Fully open
SmolVLM2 2.2B	Photo ID	Fully open
Llama 3.1 8B	Optional chat	Open weights (Llama license, gated)

Fully open matters for three reasons here:

You can share the whole box. Anyone can clone it and run it without asking permission.
You can see what it learned. For a tool people might use to look up first aid or wild plants, being able to inspect the training data is a real safety property, not trivia.
You can grow it. The next step is a first-aid and survival pack, and maybe a fine-tuned model published back to the Hub for the next person.

Everything else in the path is open too: transformers, sentence-transformers, FastAPI, and Wikipedia under CC BY-SA, credited in the app.

My Agent Session

I planned this project with Claude over the week before the stream, then built it live myself.

What the agent did:

Picked the idea with me. I asked for Hacktoberfest ideas tied to open-weight AI and chose the offline knowledge box from the list.
Turned it into a plan. That meant a day-by-day build week, a stream run of show, and a setup checklist.
Wrote a starter repo: Docker Compose with GPU and "Pi-like" tiers, a Hugging Face ingest script, a retrieval API and a benchmark script.
Made the "How open is open?" slides for the stream segment, in MLH colors.
Drafted this post from the repo I actually pushed.

What happened on stream: I threw most of the starter repo away. Running transformers natively on Windows was simpler than Docker plus Ollama on my machine, so Field Kit's code was written during the build. What survived from planning was the shape: the end state, the checkpoints, the cited-answer flow and the open-models angle.

The lesson: an agent is great at getting you to the starting line with a clear picture of the finish. The live build, the bug hunting and the judgment calls were still mine, and that's the part that made a good stream.

The Lazy Payoff

Set it up once while you're online. After that it sits on your PC and answers from your phone with no signal, no API key and no bill. The next stream moves it fully onto the phone (WebGPU + transformers.js), so you won't even need the PC.

Want to help? The README lists good first issues, from fixing a mislabeled header to getting small models to cite their sources more reliably. A Spanish Wikipedia pack would be a great first PR.

Prize Categories

None. I'm only entering the overall challenge.

Safety: Field Kit is a reference tool, not medical or emergency advice. Answers come from a small model and a partial encyclopedia, so check anything important.

DEV Community: Freddy Haug