DEV Community: Sviatoslav Barbutsa

How /search and /ask Work: Local Hybrid RAG with ChromaDB + SQLite FTS5

Sviatoslav Barbutsa — Mon, 11 May 2026 17:36:15 +0000

This is the second article in a five-part series about building Llamail, a private local AI email agent.

In the first article, I showed the whole system behind it: Gmail, Telegram, n8n, FastAPI, llama.cpp, SQLite, and ChromaDB. From the user's side it looked simple: type /search, get useful hits; ask a follow-up question in plain English, get an answer back; do it all from a phone. Quite convenient actually.

This article is about how all of that works under the hood. It will be interesting so let's dive in.

Pure semantic search is great at meaning. But ask it for exact tokens like a sender name plus invoice number, and it will often return emails that are vaguely about invoices while missing the exact identifier that mattered.

Pure keyword search has the opposite problem. It is strong on exact terms, sender tokens, and IDs, then falls apart when the query is conceptual: budget, financials, spending plan, Q2 numbers.

So I do both like this: every query goes into ChromaDB semantic retrieval and SQLite FTS5 keyword search, then the results are merged with a tiny weighted scoring function. The idea is related to rank-fusion techniques like RRF - Reciprocal Rank Fusion, which I will cover in separate article as it's quite useful, but this implementation is simpler: it merges normalized scores instead of reciprocal ranks. On my 18,000+ email mailbox, that gets me roughly ~3 seconds for search and ~7 seconds for full RAG Q&A, all on my average consumer laptop.

If you missed part 1, start there first:
From Inbox to Character: Building a Private, Local AI Email Agent

What This Feels Like from Telegram

From the outside, the flow is simple. I type /search budget and get a ranked list of emails. Then I ask something like what did John say about the budget? and the agent answers with sources. In article 1, that looked like a straightforward chat interface. Under the hood, though, those two commands are doing different amounts of work.

/search only needs retrieval. /ask runs retrieval too, but then turns the top results into constrained context for one more LLM call. The reason both commands feel useful instead of gimmicky is that retrieval is hybrid: semantic search handles meaning, and keyword search handles exact matches.

Why Single-Mode Search Breaks

Semantic search fails in predictable ways. It is good at meaning, but bad at exactness. If the user searches for an email address, an invoice number, a project code, or a sender name that really matters, embeddings can blur those details away. About invoices is not the same as from john@acme.com about invoice.

Keyword search fails just as predictably in the other direction. FTS5 is excellent at exact terms, but it has no idea that budget, financials, spending plan, and Q2 numbers are probably the same conversation. It also struggles with higher-level questions like what did John think about the proposal? unless the exact wording appears in the mailbox.

There is another failure mode that matters for RAG: aggregate questions. If someone asks how many spam emails did I get last week?, retrieval alone cannot answer that reliably, because /ask only sees the top few matching emails instead of scanning the full mailbox. The prompt explicitly tells the model to say that it only has a limited sample.

The fix is dead simple: run both search modes, normalize both score types into roughly the same range, merge them, and let the strengths of one compensate for the weaknesses of the other.

The Shape of the Pipeline

The diagram is dense a bit, so I also added a full-size version for readability:
Open full-size diagram

The bigger system still starts with email ingestion: live Gmail events or bulk import come in, the system summarizes the email and stores structured data in SQLite. In the single-email path, that same pass writes curated vector text into ChromaDB. In the chunked path, the system summarizes each chunk, creates a master summary, and writes one parent email embedding from that master summary. Search only works because the processing pipeline already did the cleanup work up front.

That is one of the reasons the Telegram experience in article 1 feels so lightweight. Most of the expensive thinking happened earlier, when the email entered the system.

What I Actually Embed

The vector side of the system is intentionally small. Instead of embedding raw emails blindly I implemented a curated representation first and then send that to the embedding model.

This is the shared parent-email embedding text builder:

# webservice/src/email_service/services/email_processor.py
def _build_embed_text(request: ProcessEmailRequest, summary: str | None) -> str:
    embed_text = (
        f"From: {request.from_name or 'Unknown'}\n"
        f"Subject: {request.subject}\n"
        f"Summary: {summary or ''}"
    )
    if request.attachments:
        filenames = ", ".join(a["filename"] for a in request.attachments)
        embed_text += f"\nAttachments: {filenames}"
    return embed_text

The key decision is that I embed the LLM summary instead the raw body. For normal emails, that is the direct summary. For long emails, it is the master summary created after chunking. That keeps the vector text semantically dense and strips out a lot of email garbage: signatures, disclaimers, nested reply chains, boilerplate greetings, and formatting noise. I also append attachment filenames so queries like the email with the budget spreadsheet still have a chance to hit.

The ChromaDB wrapper is correspondingly small:

# webservice/src/email_service/services/embeddings.py
def store(doc_id: str, text: str, metadata: dict | None = None):
    vector = embed(text)
    _collection.upsert(
        ids=[doc_id], embeddings=[vector], documents=[text], metadatas=[metadata or {}]
    )

def search(query: str, n_results: int = 5, where: dict | None = None) -> list[dict]:
    vector = embed(query)
    kwargs = {"query_embeddings": [vector], "n_results": n_results}

    if where:
        kwargs["where"] = where

    results = _collection.query(**kwargs)

And the collection setup is just:

_client = chromadb.PersistentClient(path=str(settings.chroma_path))
_collection = _client.get_or_create_collection(
    name="emails", metadata={"hnsw:space": "cosine"}
)

The embedding model is Nomic Embed v2 MoE, served by a separate llama.cpp instance on it's dedicated port. That split matters actually. Embedding requests do not block text generation on the main LLM server, so /ask and summarization are not competing for the same process which is really good for performance and the overall smooth experience.

Worth noticing that the chunked emails currently get one parent embedding from the master summary. I do not embed every individual chunk yet. That keeps the architecture simpler while still making long emails visible to semantic search. I think overengineering is one of the most common problems in software engineering and I'm definitely not immune to it, huh, so I try to keep that in mind.

The Keyword Side: SQLite FTS5

The exact-match side of the pipeline is simple SQLite FTS5. I didn't see any need for Elasticsearch, or Meilisearch, or any separate search service. Just a virtual table living next to the relational data. My general principle is: keep it simple, yet still satisfy the business requirements.

CREATE VIRTUAL TABLE IF NOT EXISTS emails_fts USING fts5(
    subject, body_text, summary, from_name, from_address,
    content='emails', content_rowid='rowid'
);

CREATE TRIGGER IF NOT EXISTS emails_ai AFTER INSERT ON emails BEGIN
    INSERT INTO emails_fts(rowid, subject, body_text, summary, from_name, from_address)
    VALUES (new.rowid, new.subject, new.body_text, new.summary, new.from_name, new.from_address);
END;

The full database setup also includes matching delete/update triggers plus a rebuild_fts() helper for one-time backfills:

def rebuild_fts():
    """Backfill FTS5 index from existing emails. One-time catch-up."""
    session = get_session()
    try:
        session.execute(text("INSERT INTO emails_fts(emails_fts) VALUES ('rebuild')"))
        session.commit()
    finally:
        session.close()

Why FTS5 is the right level of boring here:

Zero infrastructure because it's built into SQLite.
There is no additional network calls, or extra service handling, or any additional deploy.
WAL mode gives concurrent reads and writes.
For a mailbox this size, the exact-match lookup itself is cheap. The slower part of overall /search is usually local embedding inference and, for /ask, the final answer generation.

This is the FTS query:

# webservice/src/email_service/services/search.py
rows = session.execute(
    text(
        """
         SELECT e.id, -fts.rank AS score
         FROM emails_fts fts
         JOIN emails e on e.rowid = fts.rowid
         WHERE emails_fts MATCH :query
         ORDER BY fts.rank
         LIMIT :limit
    """
    ),
    {"query": query, "limit": limit},
).fetchall()

In this query, fts.rank is ordered lower-is-better, so I sort by the raw rank but expose -fts.rank as a positive score. Then I normalize it to the 0-1 range so it can be mixed with semantic scores cleanly.

One nuance: this version passes the query string directly into FTS5. That works well for normal words, but punctuation-heavy searches like raw email addresses or invoice numbers may need a small escaping or tokenization step. If FTS5 rejects the query, the code logs the failure and still returns semantic results instead of breaking the Telegram command.

The Merge That Makes It Work

The merge algorithm is the part I like most because it is so small relative to how useful it feels in practice.

In the code, the full hybrid search function is small:

# webservice/src/email_service/services/search.py
def hybrid_search(
    query: str,
    max_results: int = 10,
    account_id: str | None = None,
    after_date: datetime | None = None,
) -> list[dict]:
    semantic_hits = _semantic_search(query, n=max_results * 2, account_id=account_id)
    fts_hits = _fts_search(query, limit=max_results * 2)
    merged = _merge_results(semantic_hits, fts_hits, max_results * 2)
    return _enrich(merged, max_results=max_results, after_date=after_date)

And here is the actual merge:

def _merge_results(
    semantic: dict[str, float], fts: dict[str, float], max_results: int
) -> list[tuple[str, float]]:
    all_ids = set(semantic) | set(fts)

    SEMANTIC_WEIGHT = 0.6
    FTS_WEIGHT = 0.4

    scored = []
    for email_id in all_ids:
        s = semantic.get(email_id, 0.0) * SEMANTIC_WEIGHT
        f = fts.get(email_id, 0.0) * FTS_WEIGHT
        scored.append((email_id, s + f))

    scored.sort(key=lambda x: x[1], reverse=True)
    return scored[:max_results]

That is basically the whole trick.

Why 60/40?

Semantic search should dominate for concept-heavy questions.
FTS should still have enough weight to pull exact identifiers upward.
Emails found by both methods nearly always deserve to float to the top.
Fetching 2x the final limit from each source before trimming prevents one source from crowding out the other too early.

That's basically it and there is no fancy reranker or some fusion layer after that. Only two signal sources, a weighted sum, and a sort. Remember: simple yet functional.

A concrete scoring example makes the behavior easier to see. If email_A scores 0.90 in semantic search and 0.00 in FTS5, the combined score is 0.54. If email_B scores 0.00 in semantic search and 0.95 in FTS5, the combined score is 0.38. But if email_C scores 0.70 in semantic search and 0.80 in FTS5, the combined score is 0.74, so the result found by both systems wins.

From Ranked Results to RAG Answers

This is the point where article 1's two user-visible commands start to diverge.

/search only needs ranked results. /ask goes one step further: it takes those search hits, renders them into a prompt, adds limited conversation history, and asks the local LLM for a JSON answer.

The current prompt template is:

Answer the user's question based ONLY on the email context below.
You are Sable, a private local email agent. Be calm, factual, and concise.
If the answer is not in the context, say "I don't have enough information in the available emails to answer that."
If the question asks to count, list totals, or compute statistics across all emails, explain that you can only see a limited sample of emails, not the full mailbox.
{% if conversation_history %}

--- CONVERSATION HISTORY ---
{{ conversation_history }}
--- END HISTORY ---
{% endif %}

--- EMAIL CONTEXT ---
{% for email in emails %}

[{{ loop.index }}] From: {{ email.from_name or "Unknown" }} <{{ email.from_address }}>
Subject: {{ email.subject or "(no subject)" }}
Date: {{ email.received_at }}
Summary: {{ email.summary or "(no summary)" }}
{% if email.attachments %}Attachments: {{ email.attachments }}{% endif %}

{% endfor %}
--- END CONTEXT ---

Question: {{ question }}

Return ONLY valid JSON:
{
    "answer": "Your answer here, referencing emails by [number] when relevant",
    "confidence": "high | medium | low"
}

Conversation history is stored in SQLite via the ChatMessage table for persistence. The helper in chat_memory.py pulls recent exchanges, then trims them again against a token budget before formatting them for the prompt.

In the current code:

history is stored in SQLite
the latest rows are loaded per chat_id
history is capped at settings.chat_history_limit * 2, which works out to 20 rows by default, or about 10 user/assistant exchanges when the conversation alternates cleanly
it is trimmed again to a 2,000 token budget before prompt rendering
all Telegram exchanges are recorded, but only the ask and chitchat flows inject that history into an LLM prompt

Note: For sure I could have used LangChain or LlamaIndex with a more elaborate multi-level memory system, but that would have added a lot of complexity for little benefit in this project. A small SQLite-backed memory window was efficient for my purpose.

That is enough for useful follow-ups:

What did John say about the budget?
When was that?
Did he mention attachments?

The current context-budget reality is also quite modest:

app context window setting: 8192.
chat history token budget: 2000.
/ask search result count: 5.
there is no separate explicit max_output_tokens setting in the current ask() path; the prompt keeps the task narrow instead.

So the system stays comfortably inside an 8k context window without pretending it has full-mailbox visibility which is extremely useful on common hardware.

Small Details That Matter

The raw hybrid merge is not the whole story. Two practical details make the results feel much smarter.

Date filtering from natural language

For search-style messages, the intent classifier can turn relative time phrases into an ISO date:

"emails from last week" -> after_date: "<computed ISO date>"
"since Monday"          -> after_date: "<computed ISO date>"

The important part is not the literal example value; it is rather the mechanism. classify_intent.j2 gets today injected as a template variable, so the router can compute a real after_date before the search handler runs. In the current code, that date parameter is wired into the search flow; /ask still uses the top retrieved emails without a separate date-filter argument.

Source thresholding for /ask

The answer formatter hides weak sources:

good_sources = [(i, r) for i, r in enumerate(results, 1) if r["score"] > 0.3]
if good_sources:
    lines.append(f"Sources ({len(good_sources)}):")

That tiny cutoff fixed an annoying behavior where broad or aggregate questions would dump obviously irrelevant citations just because they happened to be in the top five retrieved emails.

Why I Like This Approach

Hybrid search is a good solution for the common RAG relevancy problems. The merge function is basically a handful of lines. But the improvement over semantic-only or keyword-only retrieval is dramatic.
SQLite FTS5 + ChromaDB, both embedded, running on one average machine, are enough to make 18,000+ emails genuinely searchable in a useful way.

In my setup that means about ~3 seconds for search and about ~7 seconds for full RAG Q&A. Slower than a cloud stack, obviously, but fast enough to be practical from Telegram on a phone. For my purposes it's perfectly fine.

If article 1 was the broad system tour, this is the subsystem that made the whole project feel real to me. Once search stopped acting like a demo and started returning the emails I actually meant, the rest of the agent suddenly had something solid to stand on.

Source code:
github.com/sviat-barbutsa/llamail

In the next article, I will move one layer up from retrieval and into the command layer: the LLM-as-router pattern that lets people talk to this system naturally instead of memorizing boring command syntax. Stay tuned!

From Inbox to Character: Building a Private, Local AI Email Agent

Sviatoslav Barbutsa — Mon, 06 Apr 2026 00:09:34 +0000

There are 18k+ emails in my personal inbox, and it's only one of the accounts I have. I wanted to search through them semantically, get AI summaries, draft replies, and run email campaigns - all from my phone. I didn't want OpenAI reading my emails or Google's AI. Or anyone's. For me, local AI is the only real answer for private data processing because no one can read your data or train their models on your data while you're paying for the service.

So I built my own - a private, local email agent project called Llamail. Its default synthetic persona is Sable. It helps me search, summarize, and manage my emails, and can also chat in a casual, roleplay-like manner to make the whole thing a bit more fun. ~3700 lines of Python, a Llama model running on my average, consumer laptop GPU, and a Telegram bot as the interface - and everything is handcrafted, not vibecoded. Here's how.

Meet Sable - a private, local email agent with a configurable synthetic persona.

Two common workflows: RAG-based search and live incoming email processing.

This is the first article in a five-part series.

Why Local?

I could've used OpenAI or Gemini or any other cloud API and been done in a weekend. But emails are personal and they contain contracts, salary discussions, medical stuff, conversations with people who didn't consent to having their words processed by a third-party AI, and I wasn't comfortable with that.

There's also the practical side: 18,000+ emails means real API costs. With GPT-5.4 pricing as of April 5, 2026 ($2.50/1M input, $15/1M output tokens), the initial import alone would likely land somewhere around $60–120 depending on average email length and summary size - not ruinous on its own, but it's still an upfront cost that scales with the size of the mailbox. Then there are search queries, draft generation, and follow-up questions - it adds up. A local LLM costs electricity and patience, but it doesn't bill you per token. And honestly, I just prefer running things locally. The independence from yet another metered subscription is its own reward.

So the constraint was: everything runs on my hardware, nothing leaves my network with one exception - Telegram. It's a third-party server and in theory they could read it if they really want to. But on paper, unlike cloud AI providers, processing my data isn't their product - they're a messaging platform, not a model training pipeline. For now that's a tradeoff I accept for the convenience of a mobile interface that works everywhere. In the future I'm thinking about implementing an adapter layer so Telegram can be swapped for other interfaces such as a self-hosted Matrix bot, a web UI, or even a plain CLI.

What It Looks Like in Practice

I control everything from Telegram. Here's a real flow:

New emails are automatically summarized and pushed to Telegram

Searching emails and asking follow-up questions with RAG

"agree but ask to clarify the refund numbers" → a polished, professional email. You say what to say, the LLM figures out how.

The agent handles search and Q&A (with simple follow-up memory), email drafting and sending (including scheduled sends), bulk Gmail import, sender blocking and unsubscribing, grammar checking, and LLM-personalized email campaigns with reply tracking.

You can use slash commands (/search budget), bare commands (import status), or just talk to it naturally - "hey, what did the team discuss last week?" The LLM reads your message, picks the right tool from 30+ available actions, extracts parameters from context (including computing "last week" into an actual date), and executes it. Between tasks, it stays in character as a configurable persona - you can chitchat with it, and it remembers your conversation history across sessions. I gave it a cold synthetic voice so it would feel like a real agent instead of a generic assistant.

Side note: Initially, the project used a persona inspired by a well-known white-haired android lady with a bob haircut. To avoid any potential copyright issues, I reworked it into an original character. It ended up being the better creative decision anyway. In practice, though, you can configure any personality you want: it is really just a prompt template, a name, and an avatar image.

It can step outside pure inbox work too, which makes the whole thing feel more like a character-driven agent than a command parser.

I'll walk through the interesting parts of how this works.

The Stack and How It Fits Together

The high-level architecture: n8n handles Gmail and Telegram events, while the Python webservice does all the actual thinking.

The system has two halves that barely know about each other:

n8n (running in Docker) is a dumb message bridge. It watches Gmail for new emails and forwards Telegram messages. That's it. The Telegram command workflow is literally three nodes: Trigger → HTTP Request → Send Message. No logic, no branching, no code nodes. For sure I could totally skip n8n and build Python services for the Gmail and Telegram APIs but I decided to save some time. Plus, n8n has a good library of connectors so if I ever need to plug in Slack, Discord, or another service, it's a drag-and-drop node instead of a new API integration so it was another solid positive argument for me.

The Python webservice (FastAPI) is the brain. Every command, every LLM call, every database query, every Gmail API interaction happens here. When I need to add a new feature, I write a Python function - not a workflow branch.

The two halves talk over HTTP. n8n lives in Docker, the webservice runs on the host. host.docker.internal bridges them.

Here's the full stack:

Component	Tool	Why
LLM	llama.cpp + Llama 3.1 8B (Q8_0)	Local, free, OpenAI-compatible API
Embeddings	llama.cpp + Nomic Embed v2 MoE (Q6_K)	Separate server so it doesn't block the LLM
API layer	FastAPI	Pydantic validation, lifespan for background threads
Storage	SQLite (WAL) + ChromaDB	One file for relational data, embedded vector store for semantic search. FTS5 for keyword search - no extra service
Orchestrator	n8n (Docker) + Cloudflare Tunnel	Gmail/Telegram triggers. Tunnel gives a free HTTPS URL for webhooks
Interface	Telegram Bot	Works from phone. No terminal, no venv, no SSH

I develop on Windows with an RTX 4080 Mobile (12GB VRAM, CUDA). Production runs on a Linux Mint mini-PC with a Ryzen 7 8845HS (Vulkan). Same Python code for both - llama.cpp handles the GPU backend difference.

The entire LLM client is 81 lines. No LangChain, no framework - just httpx.post() to an OpenAI-compatible endpoint:

# services/llm.py
def generate(prompt: str, system: str = "", json_mode: bool = True) -> str:
    url = f"{settings.llm_url}/v1/chat/completions"
    payload = {
        "model": settings.llm_model,
        "messages": [
            {"role": "system", "content": system or "You are a helpful assistant."},
            {"role": "user", "content": prompt},
        ],
        "temperature": 0.1,
    }
    if json_mode:
        payload["response_format"] = {"type": "json_object"}

    with httpx.Client(timeout=90.0) as client:
        response = client.post(url, json=payload)
        return response.json()["choices"][0]["message"]["content"]

There is no persona hardcoded into the client itself. Personality lives in the prompt templates, which makes it easy to swap or reconfigure later.

Because llama.cpp speaks the OpenAI protocol, I can swap to Ollama, vLLM, or actual OpenAI by changing one URL. The rest of the code never touches the provider. Don't get me wrong, it's not a big deal to create an adapter for any other protocol, but it's all about convenience, and I like the performance of llama.cpp. It's the only local LLM runner for me.

The One Pattern That Runs Everything

Every LLM task in the system - summarization, Q&A, intent classification, drafting, campaign personalization, reply classification, grammar checking and even a casual conversation - follows the exact same pattern:

Jinja2 template → llama.cpp → parse JSON

There are 11 templates. Each one is a contract: "here's what I'm giving you, here's the JSON structure I expect back." I found that the template system works perfectly for setting LLM behavioral boundaries and expectations, simply because they're separated instruction files that support variable syntax inline and you keep all your instructions in a clean, isolated space.

Here's the email summarization template:

Analyze this email and return ONLY valid JSON.

From: {{ from_name or "Unknown" }} <{{ from_address }}>
Subject: {{ subject or "(no subject)" }}
Date: {{ date }}

--- EMAIL BODY ---
{{ body }}
--- END ---

Return this exact JSON structure, nothing else:
{
    "summary": "2-3 sentence summary",
    "category": "work | personal | newsletter | finance | spam | other",
    "priority": "high | medium | low",
    "sentiment": "positive | negative | neutral | urgent",
    "action_required": true or false,
    "action_items": ["action item 1"] or [],
    "key_people": ["Person name"] or []
}

The same pattern handles intent classification (user types "find emails about the budget" → LLM returns {"intent": "search", "params": {"query": "budget"}}), draft generation (instructions + original email → LLM returns the draft text), and everything else. Same generate() function, different template.

This is the decision I'm happiest with. Adding a new LLM-powered feature means writing a Jinja2 file and a thin Python function. No new abstractions, no plumbing. In general, the "hyped AI skills" the whole internet has been buzzing around lately are nothing but text-based instructions backed by tools. So that's it - we added "skills" to our agent in the most convenient way.

The Three-Tier Command Router

This is the most interesting pattern in the codebase. When you send a message, the system tries three strategies - fast to slow:

Tier 1: Slash commands. /search budget → instant. The system splits on whitespace, looks up the command, and dispatches. Zero LLM involvement.

Tier 2: Bare compound commands. import status or draft reply 1 sounds good (no slash) → also instant. These are safe to match without the LLM because they always start with a known keyword (import, draft, campaign, schedule).

Tier 3: Natural language. "hey, how's my import going?" → the agent sends you an "Analyzing your message..." notification (so you know it's working), passes your text to the LLM with a classification template listing 30+ intents, gets back structured JSON, and dispatches to the right handler. So instead of passing all the instructions directly into one LLM call and hoping it will process everything at once correctly, I use separation of concerns (yes, good old SOLID): first, the agent classifies the user's intent and nothing more. Only after it correctly picks the appropriate action - including whether further LLM processing is needed - does it pass the intent data back to the system, which executes the next step. That might include further agent processing such as a roleplay conversation or dedicated tool usage. This approach allows smaller models like 8B or even lower to correctly handle such tasks without any hassle.

def handle_command(text: str, chat_id: str | int = "") -> str:
    # Tier 1: /slash commands
    if text.startswith("/"):
        command = text[1:].split()[0].lower()
        ...

    # Tier 2: bare compound commands
    else:
        first_word = text.split()[0].lower()
        if first_word in {"import", "draft", "campaign", "schedule"}:
            command = first_word
            ...

    # Tier 3: LLM fallback
    if not command:
        return _llm_route(text, chat_id)

The key: the LLM is the fallback, not the primary path. Slash commands have zero overhead and it's a really convenient way for people who clearly know what they want and just want to get the result without any blabbering:
command -> result -> done. Natural language adds ~4 seconds - and only when there's no other way to understand the input. You get the convenience of free-form chat without paying the latency cost on every message.

Things That Broke

These are the bugs I spent the most time on. If you're building something similar, maybe this saves you a few hours.

1,628 failed imports from one bug. I was importing 18,000 emails, and 1,628 kept failing with foreign key constraint errors. The cause: I was saving email chunks before the parent email row existed. The fix was literally moving one function call above another. The lesson was less fun - I should have had integration tests from the start. No, I still don't have them, and yes, I know I could ask Codex or Claude to auto-generate some, but that would break the whole handcrafted, no-vibecoded spirit of the project. If you're going to be honest, you might as well be honest all the way through.

Gmail IDs are lies. The Gmail API returns hex IDs like 19c54cb15118c128. The Gmail web UI uses completely different IDs like QgrcJHrtw.... I spent an afternoon trying to build a "View in Gmail" link. Turns out there is no conversion between these two formats. No API, no formula, nothing documented. I ended up using rfc822msgid: search links - the RFC822 Message-ID header is in every email, and Gmail search can find by it. It opens a search with one result instead of a direct link, but it works. Mostly.

Google OAuth and the trailing slash. http://localhost:9090 and http://localhost:9090/ are different redirect URIs to Google. One works. One gives you redirect_uri_mismatch. I tried the wrong one first and spent 10 minutes reading Stack Overflow threads about client ID misconfigurations before noticing the slash.

Telegram eats angle brackets. I had "Usage: import history <account_email>" in a response. Telegram's HTML parser tried to interpret <account_email> as an HTML tag, failed to find a closing tag, and silently dropped the entire message. This happened twice - once with static text, once with email content containing <sender@email.com>. The real fix was html.escape() at the route level so I'd never have to think about it again.

Real Numbers

Operation	Time
Process one email (summarize)	~2-3.5 s
Process one email (embedding)	< 0.1 s
Bulk import throughput	~5 emails/minute
Hybrid search (semantic + keyword)	~3 s
RAG Q&A (search + generate answer)	~7 s
Natural language intent classification	~4 s
Slash commands	Instant

All numbers measured on real data - bulk import throughput from SQLite job timestamps (started_at to finished_at), LLM call times from llama.cpp server logs, everything else with a stopwatch and a Telegram chat (yes, a very scientific method).

Hardware: RTX 4080 Mobile (12GB VRAM), 32GB RAM. The 8B model in Q8_0 quantization uses about 9GB VRAM. You could run Q4 on a 6GB card, but response quality drops noticeably for structured JSON output.

Try It

Source code: github.com/sviat-barbutsa/llamail

You'll need:

A GPU with 8GB+ VRAM (NVIDIA with CUDA, or any Vulkan-capable card)
16GB+ RAM
Docker for n8n
A Gmail account and a Telegram bot (both free)

The repo has a setup guide that walks through everything.

In the next article, I'll go deep into the hybrid search system - how combining ChromaDB semantic search with SQLite FTS5 keyword search produces better results than either one alone, and why it only took 120 lines to build.

If you're building something with local LLMs, I'd love to hear about it in the comments. And if you've solved the Gmail ID problem more elegantly than I did, please tell me.

P.S. The bot avatar shown in the screenshots was generated locally, and the Sable persona is original to this project.