DEV Community

mycute
mycute

Posted on Edited on

Atom Zero: a 49M local assistant that says "I don't know"

Most local AI demos start with a bigger model. I went the other way: a 49.2M parameter model trained from scratch, wrapped in a runtime that does the things a small model is bad at — retrieval, tools and honest missing-source handling.

The problem with small models

A 49M model is not a general assistant. On a no-RAG knowledge probe it scores 2/10. It will not write your backend or answer MMLU. Pretending otherwise is how tiny-model demos lose trust.

So Atom Zero separates responsibilities:

  • the model handles short wording;
  • SQLite + hybrid FTS/MiniLM retrieval supplies facts and citations;
  • deterministic tools handle arithmetic, tasks, time and memory;
  • a reviewed LoRA path handles style/skills, not facts.

What it actually does

  • Retrieves from your own UTF-8 documents and shows source, title and sha256.
  • Says "I do not have a source for that yet" instead of inventing an answer.
  • Records unanswered questions as knowledge gaps.
  • Turns gaps into pending curiosity topics; a human approves before any fetch or teacher call.
  • Keeps personal notes, tasks and a browser session locally.
  • Optionally routes free-form chat to a local Ollama/OpenAI-compatible provider while RAG and tools stay local.
  • Runs with no telemetry and no hosted service.

Benchmarks, with the important caveat

check result
raw 49.2M, no RAG/tools 2/10
product with RAG/tools 21/21
behavior checks 16/16
source-cited retrieval top-1 1.000 (FTS 0.850)
tests 412 passed

The 21/21 is the product, not the bare model. The benchmark is narrow and authored: tools, memory, retrieval, missing-source behavior and one short generation. It is not MMLU, GSM8K or HumanEval.

Curiosity without autonomous weight mutation

The loop is deliberately boring:

question -> no local source -> knowledge gap
gap -> pending curiosity topic
human approves
worker fetches Wikipedia context, local Ollama teacher synthesizes
answer stored as an **unverified teacher note** in RAG
next answer can cite it; weights_changed: false
Enter fullscreen mode Exit fullscreen mode

A live run produced unapproved_run: 0, then approved_run: done, then a cited answer. The model did not decide what to study and did not mutate its own weights. Weight updates happen only through reviewed feedback -> gated LoRA -> explicit activation/rollback.

Try it

pip install atomzero==0.1.6
atomzero-quickstart
Enter fullscreen mode Exit fullscreen mode

Try these:

What is the launch code?                         -> cited local answer
What is the population of Paris?                 -> no source; gap recorded
Propose curiosity topics                         -> pending, approval required
Add task buy milk / My name is Alex              -> local tools + memory
Enter fullscreen mode Exit fullscreen mode

Optional stronger local chat:

atomzero-quickstart --provider
Enter fullscreen mode Exit fullscreen mode

What it is not

  • Not a ChatGPT competitor or a general reasoning model.
  • Not a huge model pretending to be small.
  • Not a hosted service; it binds to localhost by default.
  • Not a system that silently crawls or trains on your data.

Links

If you run local models on a small GPU, want source-cited notes, or are building a compact RAG/agent reference, I would like to hear what breaks first.

I am open to AI/LLM Engineer roles with EU relocation or remote.

Apache-2.0 for code and weights. Built on a 6 GB RTX 3050 laptop.

Full project map

The canonical end-to-end map is here:

https://github.com/byjameson/stream-finetune/blob/main/docs/PROJECT_OVERVIEW.md

It covers the 49.2M model and 40k training run, RAG (FTS + MiniLM, prewarmed matrix, IVF ANN + exact fallback), deterministic tools, sessions/export/delete, approved research and the Ollama teacher, the curiosity approval gate, the RAG -> reviewed LoRA -> full-training ladder, the optional local provider, the optional safety filter, privacy/security, every benchmark with caveats, packaging and limits.

Contact: gokhancava@gmail.com · GitHub · LinkedIn

Top comments (0)