I was one commit away from putting a vector database behind my AI's voice. I deleted that plan and used a single markdown file instead. The drafts got better, and the context now costs me $0 a month.
The feature is a Note drafter I built into my Substack growth dashboard. You give it a prompt like "build-in-public milestone, 3 variants" and it returns three Substack notes that sound like me. What made it worth writing up was where the complexity actually belonged, which turned out to be somewhere much simpler than I first assumed.
The first design I sketched stored every published note and every voice decision in a vector database, and let the model retrieve the relevant pieces per draft. Embeddings and a similarity search in front of the model, the standard RAG pipeline. That setup is built for fetching knowledge on demand, which sounded right until I looked at what voice actually needs.
What voice actually is
Voice for an AI assistant is a stable identity: who you are and what tone you use. It is the part that should come out the same every single time you draft.
Knowledge changes by query. What happened recently in this project, or what a term means: that kind of context shifts per question and benefits from retrieval.
For a Note drafter specifically, consistency is most of the value. The whole point of the feature is that three drafts in a row read like the same person wrote them, and that the person is me. If the voice drifts between drafts, I have to rewrite them by hand, which defeats the reason I built the tool. So the register is the one thing I cannot let move.
RAG is a knowledge tool. When you bolt it onto a voice problem, a few things break, and they all push the output in the same wrong direction.
The context stops being consistent. Two adjacent prompts pull different chunks, the model sees different priors, and the voice wanders between drafts. One note sounds like me, the next sounds like a press release.
The retrieval also pulls in irrelevant hits. A query about a "build-in-public note" matches old project facts and architecture decisions. None of that is voice. It is noise, and the model folds it in, dragging the register toward generic blog post.
And you pay for all of it. Each generation turns into a vector search call plus an LLM call. For voice, the search half adds latency and cost and returns nothing you needed.
RAG pulls different chunks each query; a voice baseline stays constant. | Generated with Claude
What I did instead
The voice lives in a 500 to 1000 token markdown file in the database, edited from a settings page. Nothing clever. It looks roughly like this:
# Voice profile
## Tone
- Specific, no fluff. Short sentences.
- Czech directness in English. Admit uncertainty when present.
## No-go phrases
- "In today's fast-paced world"
- "Game-changer", "Unlock", "Mind-blown"
- Em-dashes used as a drama device
## Good note examples
[3 actual high-performing notes pasted in]
## Bad note examples (and why)
[1-2 generic AI-sounding notes pasted in]
On top of that file I inject the last 10 published notes from the app's own database as a register reference. They are not retrieved by similarity. They are the trailing window: the last 10 published notes, always that same set, pulled without any search step.
Assembling that window is a plain SQL query. The dashboard already stores every note I publish, so the drafter just selects the ten most recent rows ordered by publish date and pastes their text into the prompt. There is no embedding step and no index to keep warm. The query runs in a few milliseconds and the result is the same for every prompt in a given moment, so it stays cheap no matter how many drafts I ask for.
That trailing window matters more than it looks. It keeps the model anchored to how I have actually been writing lately, not how I wrote six months ago, and it updates on its own as I publish. The file holds the rules, and the window keeps showing the model how I write right now.
Editing it is just as boring, which is the point. When a draft comes out wrong, I open the settings page, add the offending phrase to the no-go list or paste in a better example, and save. No re-indexing, no pipeline to babysit. The next draft picks up the change immediately.
The whole prompt lands at about 2k tokens in and 300 out, on Gemini 2.5 Flash Lite. That works out to roughly $0.0003 per draft, or about five cents for a month of five drafts a day.
The 80% comes from the examples
The strongest thing I took from this build: three good note examples plus one or two bad ones did more for output quality than a model upgrade would have.
The model learns the register from actual sentences, not from a description of the register. A rule like "no fluff phrases" tells it what to avoid. An example shows it the target to hit, and a target is far easier to copy than a prohibition.
I tested it both ways. With rules only, a "shipped a new feature" draft came back as something like "Excited to share a game-changing update that will supercharge your workflow." Correct topic, generic voice, exactly the register I do not use. With three of my real notes in the file, the same prompt produced a draft that opened with the concrete thing I had shipped and why it annoyed me enough to build it, in short flat sentences. Same rules, same model, same prompt. The only difference was that the model now had sentences to imitate instead of a list of things to avoid.
Rules alone drift generic; three real examples pull the draft into voice. | Generated with Claude
I collect more of these "where does the complexity actually belong" calls in a free email series, The Claude Code Memory Starter.
When RAG is right
None of this is anti-RAG. Retrieval earns its place when the context actually changes by query, so that feature X and feature Y pull different docs, or when there is more context than the prompt window can hold.
There is a spot for it in this same dashboard, in fact. The day I add a "what have I already written about pricing" lookup, that is retrieval work: the answer is a specific past post, and there is far more of it than I want sitting in a prompt. The voice file has nothing to do with it.
For voice, none of that is true. The identity is the same every time, and it fits in the prompt with room to spare. Getting the register right is a matter of showing the model the right examples, not searching for them.
The rule I keep coming back to
Voice is a baseline. Knowledge is retrieval.
Bake the constant part into the system prompt as a static blob, and retrieve only what actually changes per query.
I built this for my own dashboard, but the same split holds for any AI feature you bolt onto your own tool, whether that is a chat wrapper or a content drafter like mine. Before reaching for a vector DB, ask whether you actually need retrieval, or whether you just need a baseline.
Most of the time, you do not.
External Sources
- Retrieval-augmented generation — what RAG is, in one page
- Pinecone — managed vector database for retrieval
- Google AI for Developers — Gemini Flash Lite models and pricing
I write up more of these build decisions as a free email series, The Claude Code Memory Starter.
I'm Daniel. I build content tools and small games solo, and write about what breaks on the way.
I build small tools and kits for solo creators. You can find them here: https://danielrusnok.gumroad.com
Top comments (0)