DEV Community

Ai2th
Ai2th

Posted on Edited on

Miktam Bible | Part 2 — An AI Study Assistant That Runs on the Phone

Most AI features in apps are a thin wrapper around a cloud API. Miktam Bible's study assistant is different: the language model runs on the phone, and every answer is grounded in verses pulled from the offline database.


🧠 The model

The assistant runs SmolLM2-135M, a small model quantized to 4 bits in GGUF format (about a 105 MB one-time download), through llama.rn, the React Native binding for llama.cpp. The settings are chosen for phones:

context = await initLlama({
  model: nativePath,
  n_ctx: 2048,        // Mobile-safe context window
  n_batch: 256,       // Mobile-safe batch size
  n_threads: 4,       // CPU threads for inference
  n_gpu_layers: 0,    // 0 = CPU only
  use_mlock: false,
  use_mmap: true,     // Memory-map the model file
});
Enter fullscreen mode Exit fullscreen mode

CPU-only with a memory-mapped model file keeps it working across the wide range of Android hardware. If the model has not been downloaded yet, or fails to load, the chat shows a banner and switches to a limited offline mode with pre-written study answers. Download the model for real, verse-grounded answers.


📚 Answers grounded in Scripture (RAG)

A small model on its own will happily make things up. So every question goes through a retrieval step first:

Your question
  └── 1. Search the Bible (FTS5, with synonym expansion and stop words removed)
        └── 2. Take the 10 most relevant verses
              └── 3. Build the prompt: those verses + your question, in the template the model was fine-tuned on
                    └── 4. Generate the answer on the phone
                          └── 5. Turn verse references in the answer into tappable links
Enter fullscreen mode Exit fullscreen mode

The model sees the retrieved verses as context for your question, and any reference in its answer (for example John 3:16) becomes a link back into the reader.


🔮 What's next: hybrid search

Keyword search finds verses that share words with your question, not verses that share its meaning. The pipeline is already shaped for hybrid search: FTS5 results plus vector similarity (sqlite-vec), merged with reciprocal rank fusion. The missing piece is a small on-device embedding model (MiniLM); until it lands, the assistant uses FTS5 alone.


🔒 Private by design

No API keys and no server: your questions are never sent anywhere. Once the model is downloaded, you can put the phone in flight mode and the assistant still works.


Next, Part 3: reading plans and reminders that keep up with you.

📥 Get Miktam Bible

Miktam Bible is in Google Play closed testing:

  1. Join the AI2TH Google Group with the Google account on your phone. Members are testers for every AI2TH Play app.
  2. Opt in at play.google.com/apps/testing/miktam.bible.
  3. Install from Google Play.

Different Play account? Request access here.

Top comments (0)