I'm Building Free AI on a VM That Barely Qualifies as a Computer

How I am using small models, BM25, RRF, response contracts, relationship validation, recovery loops, and a brutally measured free compute budget to build Scout.

I am building an AI system on a budget best described as please do not create a billing event.

That is not a figure of speech. I have zero dollars for this and no credit card to attach to a "free" promotion that becomes a surprise archaeology project six months later.

Not a wrapper that sends every thought to a premium model. Not a demo that works until the free trial expires. Not a local setup where I announce that inference is "free" while my own computer quietly converts electricity into heat and resentment.

I mean a cloud-hosted assistant designed to run inside free-tier constraints, answer real people, remember a conversation, use tools, retrieve verified facts, reject false claims, recover from bad generations, and remain portable enough that I can eventually replace my information with another business's knowledge package.

Its name is Scout.

Scout started as an embeddable recruiter chatbot for my portfolio. It is now becoming something much larger: a domain-neutral intelligence engine that can power multiple assistants without rebuilding the core every time. A tire shop, a SaaS company, or a recruiter assistant should be able to change the identity, personality, tools, and knowledge while keeping the same retrieval, memory, validation, recovery, telemetry, and evaluation machinery.

The published work lives on my public feat/agent-systems-network branch. At the time I wrote this, the remote branch head was 952646e, 149 commits ahead of develop, and it was still deliberately not in production. The newest Cloudflare qualification work described later is local and intentionally unpushed while I verify the implementation around the accepted measurement baseline.

That last sentence matters. This is a build report, not a victory lap.

First, what do I mean by "free AI"?

I do not mean that compute has stopped obeying economics.

I mean the system is designed so the user does not need a paid hosted-LLM subscription or a pile of expiring promotional credits. The target is cloud-hosted inference running inside a measured free allowance, with as much inspectable intelligence as possible moved into Scout's retrieval, state, tools, contracts, validation, and recovery system.

Google Cloud's current Free Tier documentation says eligible e2-micro use is free each month up to the number of hours in that month, combined across supported regions. It also explicitly says GPUs and TPUs are not included. Read the actual Google Cloud Free Tier rules before assuming the word "free" is a legally binding force field.

An e2-micro is a shared-core machine with roughly 0.25 vCPU and 1 to 2 GB of memory according to the Compute Engine machine documentation. In technical terms, it is not a beast. It is more like a polite suggestion of a computer.

One development and evaluation path uses qwen2.5:1.5b through Ollama. Qwen's official model card lists 1.54 billion parameters and an Apache 2.0 license. The model supports a much larger theoretical context, but Scout intentionally operates inside a far smaller budget because the machine does not care what the brochure says. You can read the official Qwen2.5 1.5B model card and my pinned runtime metadata.

One more important correction: Ollama is a development and qualification torture chamber. It is not Scout's product identity, and local inference is not magically free because the electric meter is somewhere else.

The newest hosted qualification path uses Cloudflare Workers AI with @cf/meta/llama-3.2-3b-instruct. As of August 15, 2026, Cloudflare's pricing page lists a free allocation of 10,000 neurons per day, resetting at 00:00 UTC. The same page lists the 3B model at 4,625 neurons per million input tokens and 30,475 per million output tokens. Read the actual Workers AI pricing rules because free tiers are product rules, not promises from the universe.

The current production chatbot still uses a Groq-hosted Llama 8B model that is being retired. Local Ollama, Cloudflare qualification, another hosted provider, and future browser WebGPU are interchangeable implementation choices behind Scout's model boundary. The intelligence engine should survive all of them. That distinction is documented in the branch's current feature handoff.

So the honest pitch is this:

I am building an AI system with no paid-model subscription, designed to stay useful inside a measured free allowance when the workload and provider rules allow it. I am not claiming I discovered infinite silicon behind a gas station.

The model is not the system

My original mistake was the same mistake I see everywhere: treating the model like it should do every job.

People throw a giant prompt at an LLM and expect it to be:

• the database
• the search engine
• the conversation state
• the planner
• the tool router
• the fact-checker
• the security boundary
• the copywriter
• and, somehow, the QA department

Then the model confuses two projects and everybody declares that small models are useless.

Small models are not useless. They are just terrible unpaid interns when you give them nine jobs, no source of truth, and a prompt that looks like a CVS receipt.

Scout works differently.

The model has one main responsibility: turn a constrained packet of verified meaning into natural language.

Everything else is software.

The current request path is:

1. Reject prompt injection, private-data requests, and requests to invent credentials before inference.
2. Normalize the query, protect important technology names, correct typos, classify intent, and resolve conversational references.
3. Retrieve verified evidence with BM25. For context-dependent follow-ups, fuse several lexical views with Reciprocal Rank Fusion.
4. Run allowlisted, read-only evidence tools when the question needs comparison, skill evidence, role matching, or a recruiter brief.
5. Build a response contract that fixes the required entities, answer polarity, evidence strength, boundaries, and forbidden claims.
6. Ask the 1.5B model to phrase the answer.
7. Extract and validate factual claims, relationships, technologies, numbers, polarity, relevance, length, and safety.
8. If the draft fails, generate a repair using the exact rejection reasons.
9. If that fails, generate from a stricter recovery contract.
10. Record the route, latency, model, cost-adjacent events, recent turns, and topic stance.

That is the real trick. I am not making the model bigger. I am making its job smaller.

Scout started as one script tag

The frontend is intentionally boring in the best possible way. Scout can be embedded with one script tag:

<script src="https://bradleymatera.github.io/ProjectHub/ProjectHub.js"></script>

The widget is vanilla JavaScript and GitHub Pages friendly. The backend is Node and Express. The knowledge is bundled JSON. The current branch is Dockerized so development can match the eventual cloud runtime more closely. The high-level structure and file map are in the branch README.

This began as a recruiter assistant that answered questions about my projects, AWS internship, education, skills, honest gaps, and target roles. That bounded domain turned out to be useful. It gave me something most AI demos do not have: a knowable truth set.

If Scout says I built ProjectHub during my AWS internship, that is not a philosophical disagreement. It is wrong. ProjectHub and the internship are both real, but the relationship is false.

That kind of mistake became the center of the architecture.

Step 1: Build a real source of truth

Scout does not ask the model to remember my life from training data. The source of truth is data/recruiter-knowledge.json, plus project metadata and approved source material.

The knowledge package contains things like:

• identity and assistant persona
• projects and their actual technology stacks
• employment and internship facts
• certifications and education
• claims Scout may make
• claims Scout must never make
• honest boundaries and gaps
• source provenance

That knowledge gets flattened into retrievable chunks. The generator and the validators consume the same canonical facts. If a fact is not in the package, Scout does not get to manufacture it because the sentence would sound nicer.

This is also what makes the engine sellable later. My name cannot be hidden inside 500 if (Bradley) statements like a cursed family heirloom. Identity and knowledge must be data. Core retrieval, planning, validation, and recovery must remain generic.

The latest branch commit proves that direction. The technology-claim validator was rewritten to contain no hard-coded person names, project names, or benchmark-specific literals. Its tests use unrelated fixtures such as Maria Lopez, Nebula Engine, Carlos Rivera, and Quantum Portal to prove the same engine works without my identity. See the portable validator commit and the synthetic test suite.

Step 2: Understand the question before retrieving anything

Retrieval cannot save a query you misunderstood.

Scout's query-understanding pipeline does five cheap things before generation:

function understandQuery(query, history, chunks) {
  let normalized = normalizeQuery(query);

  if (chunks && chunks.length > 0) {
    const vocab = buildVocabulary(chunks);
    normalized = correctTypos(normalized, vocab);
  }

  const expanded = expandQueryAliases(normalized);
  const intent = classifyIntent(expanded);
  const rewritten = rewriteQuery(expanded, history);

  return { original: query, normalized, expanded, intent, rewritten };
}

That is a shortened excerpt from the real lib/query-understanding.js.

Typo correction uses Damerau-Levenshtein distance against vocabulary derived from the knowledge chunks, but protected terms are skipped. That protection exists because a "helpful" typo corrector once sees something like COBOL and decides it must be a different word. The actual correction logic is here.

Then the conversation resolver builds state from recent turns:

• active entity
• previous entity
• comparison entities
• topic scope
• last intent

It can rewrite "What did he use there?" into a question about the active company or project. It can resolve "the other project" from a previous comparison. If no referent exists, it refuses to invent one. The generic resolver is in lib/conversation-resolver.js.

I learned this the annoying way. If you wait for a 1.5B model to solve coreference after retrieval, it can confidently answer the wrong subject using perfectly real evidence. That is a very sophisticated way to be wrong.

Step 3: Use BM25 because boring algorithms still work

Scout's first-stage retriever is a dependency-free Okapi BM25 implementation in JavaScript.

I did not invent BM25. I am not about to grow a mustache, stare into the middle distance, and imply I discovered probabilistic information retrieval in my garage. Robertson and Walker were publishing the foundation decades ago. A useful overview is The Probabilistic Relevance Framework: BM25 and Beyond.

BM25 works well here because the corpus is small, verified, and full of meaningful exact terms such as project names, AWS services, certifications, and technologies. I do not need a paid vector database to find DynamoDB in a few hundred knowledge chunks.

Scout uses k1 = 1.2 and b = 0.75:

score += idf * (f * (this.k1 + 1)) /
  (f + this.k1 * (1 - this.b + this.b * (dl / avgdl)));

The full implementation builds term frequency, document frequency, smoothed inverse document frequency, document-length normalization, and top-k scoring in about 120 lines. Read lib/bm25.js.

On the checked-in 40-query golden retrieval set, Scout achieved:

• Recall@6: 1.000, meaning the expected evidence appeared in the top six for all 40 queries
• MRR@6: 0.971, meaning the first correct result was usually ranked very near the top

The queries are public in data/eval-golden.json, and the evaluator that calculates Recall and Mean Reciprocal Rank is in scripts/eval-retrieval.js. The exact results are recorded in the feature handoff.

Why I did not replace everything with embeddings

Dense retrieval is useful. It is not automatically the answer to every corpus.

Embeddings would add model memory, startup time, index management, and another failure surface to a machine already fighting for oxygen. BM25 is fast, inspectable, local, and excellent at exact technical vocabulary.

Free AI engineering is mostly the art of refusing dependencies you cannot prove you need.

Step 4: Use RRF only where it actually helps

A follow-up question can have several useful query views:

• the literal message
• an alias-expanded version
• a version rewritten with conversation context

Scout runs BM25 for each useful view, then combines the ranked lists with Reciprocal Rank Fusion:

RRF(d) = sum over each ranking r: 1 / (k + rank_r(d))

The implementation uses the standard smoothing constant k = 60:

const rank = index + 1;
current.rrfScore += 1 / (60 + rank);

That code is in lib/rrf.js. The original technique comes from Cormack, Clarke, and Büttcher's 2009 paper, Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods.

Here is the part I care about most: I tested it instead of worshipping it.

Applying several correlated RRF views to every standalone query lowered Scout's offline MRR from 0.971 to about 0.942. So I rejected global RRF. Standalone questions keep the strongest direct BM25 ranking. Contextual follow-ups get fused views when history makes those views genuinely different. That experiment and decision are documented in the active feature handoff.

Algorithms are tools. If the number gets worse, the algorithm does not get to stay because it has a cool acronym.

Step 5: Make the model obey a response contract

Retrieval gives Scout evidence. It does not guarantee a useful answer.

For each question, lib/response-contract.js builds a semantic contract containing:

• intent and sub-intent
• active and required entities
• the requested topic
• ranked key facts
• required relationships
• answer polarity such as YES, NO, MIXED, FIT, PARTIAL_FIT, or NOT_FIT
• evidence strength such as project, internship, certification, or professional experience
• an honest boundary that must be mentioned
• forbidden claims
• a target response shape

The important idea is that the model does not decide whether the real answer is yes or no. The harness determines polarity from verified evidence first. The model decides how to say it naturally.

For example, a question like "He was a senior AWS engineer, right?" should not become a creativity exercise. The contract can require NO, require the AWS internship entity, require the entry-level boundary, and forbid title inflation.

The real direct-answer logic is visible in determineDirectAnswer, and the contract turns that state into normal language instructions in buildNaturalInstructions.

This is one of the biggest ways to get more from a small model. Do not ask it to discover the truth and write the sentence in the same breath. Hand it the truth in a form that is hard to misunderstand.

The worst bug: two true facts can make one false sentence

Early validators checked whether the answer contained known entities and source words. That sounds reasonable until the model says something like:

ProjectHub was built during Bradley's AWS internship.

Every noun in that sentence is real.

The sentence is still false.

An earlier manual audit of 46 answers that the automated system had accepted was brutal:

• 12 were clean or mostly clean
• 28 were factually wrong through overclaiming or wrong entity relationships
• 4 were non-answers
• 2 were borderline

That audit is public in data/accepted-answer-audit.md. It was not fun to read. It was also more valuable than another week of changing prompts.

The root problem was that grounding cannot stop at entity existence. It has to validate relationships.

So Scout now builds a graph of subject-relation-object triples with provenance:

ProjectHub -> uses_tech -> Node.js
AWS capstone -> uses_tech -> DynamoDB
Bradley -> interned_at -> AWS
ProjectHub -> built_during -> AWS internship   // absent, therefore unsupported

The graph builder is lib/relationship-graph.js. Generated text is converted back into normalized claim classes by lib/claim-extractor.js. Then lib/relationship-validator.js checks whether each claimed relationship is actually supported.

This catches a model recombining unrelated truths. It also handles negation, because "he did not attend MIT" should not be rejected merely because MIT is absent from the knowledge graph. Refuting a fabricated premise is correct behavior.

Technology claims need their own validator

Technology names are especially annoying.

They contain punctuation. They have aliases. They are sometimes normal English words. Node.js can become NodeJS. Go should not match Google. A sentence can say "does not use Express but uses React," where one technology is negated and the other is a positive claim.

The current branch adds a dedicated lib/tech-claim-validator.js. It:

• extracts technologies only inside claim contexts such as "uses," "built with," or "has experience with"
• splits positive and negated clauses
• canonicalizes punctuation and casing
• resolves safe aliases
• compares normalized tokens on both sides
• rejects unsupported technologies
• avoids unsafe substring matches such as Go inside Google
• contains no Scout-specific identity literals

The core normalization is simple:

function canonicalize(value) {
  return String(value || '')
    .toLowerCase()
    .replace(/[^a-z0-9]/g, '');
}

The hard work is not that function. The hard work is deciding when a token is a factual technology claim, handling clause-level negation, and proving portability with unrelated test fixtures. The latest implementation and tests landed in 952646e.

Generation, repair, and recovery all share one deadline

Scout's LITE path is not "send prompt, hope, return string."

runLiteAgent owns a request-scoped deadline. Every generation attempt asks how much time remains. The model adapter links that deadline to an AbortController, so a slow request can actually be cancelled instead of politely continuing after the user has left. See lib/local-model-router.js.

The recovery sequence is:

1. normal evidence-bound generation
2. targeted repair using validation failures
3. strict recovery generation from a semantic contract
4. a minimal generated evidence boundary if the model still cannot produce a valid answer

The deterministic system can build the contract. It is not supposed to write the final conversational prose. That is the branch's current architectural transition and one reason the release gate still says NOT YET.

That rule exists because an earlier audit found that 41% of the visible answers were still deterministic final output. The system looked safe, but Scout was sometimes passing a benchmark by printing canned prose instead of getting the model to cooperate. Later work reduced the reachable deterministic conversational prose writers to zero. The hosted baseline described below then verified the architecture again: every successful visible answer had a tracked model call.

Repair prompts include human-readable reasons such as:

• answer the actual question directly
• remove exaggerated language
• name the required entity
• preserve the NO polarity
• do not claim this technology without evidence
• write one or two complete sentences

That is much more useful than telling the same model "try again but better," which is the AI equivalent of a manager leaning into your cubicle and whispering synergy.

A one-line configuration bug ate four seconds

The model was configured with a 1,536-token context in the primary path, while repair and recovery inherited 2,048. Ollama reloaded the model when the context size switched, costing roughly four seconds on the test machine and blowing the 15-second request budget.

The fix was one line:

- OLLAMA_AGENT_CONTEXT=2048
+ OLLAMA_AGENT_CONTEXT=1536

That commit is public: c5656e8.

This is why I keep saying the harness matters. Sometimes "AI performance engineering" is a new algorithm. Sometimes it is discovering that two environment variables are fighting behind a dumpster.

The expensive part was not writing the answer

The 4-vCPU CPU-only profile produced one of the most useful measurements in the entire build:

Model work	Measured time	Measured size
Prompt evaluation	about 4.1 to 5.6 seconds	about 285 to 326 tokens
Actual answer generation	about 0.4 to 0.6 seconds	about 13 to 18 tokens

The model could write the little answer quickly. The expensive part was feeding it enough context to understand what Scout wanted.

One successful pass was roughly six seconds. A rejected first draft plus repair was roughly twelve. Add recovery and the path drifted toward eighteen seconds, which violates the 15-second product contract. I rejected an attempted "improvement" that changed the deadline to 25 seconds. Making the clock more patient does not make Scout smarter.

At the c5656e8 checkpoint, the 13-case qualification result was 4 GOOD and 9 inference deadlines. It also showed zero deterministic prose, zero semantic-safety errors, zero routing errors, and zero polarity errors. That did not prove the intelligence engine was wrong nine times. It proved the tiny CPU runtime could not reliably complete generation, rejection, repair, and recovery inside the real deadline.

That gave me two separate engineering tracks:

1. Improve first-pass semantic packets so the model produces a valid answer before recovery becomes necessary.
2. Qualify enough hosted compute for the legitimate cases that really do need a second model pass.

The first track makes Scout more efficient everywhere. The second stops me from pretending a quarter-core VM can be negotiated with emotionally.

Tools are deterministic and read-only

Scout has allowlisted tools for evidence-heavy operations:

• search the portfolio
• fetch a specific project
• compare projects
• match verified experience to a role
• retrieve skill evidence
• build a recruiter brief
• retrieve the public candidate profile

The definitions and execution boundary are in lib/agent-tools.js.

These tools cannot execute arbitrary commands, write files, send messages, or browse random websites. Unknown tool names fail closed.

This is another place where "agent" gets overcomplicated. The model does not need root access to feel intelligent. It needs the smallest set of trustworthy operations that answer the domain's real questions.

I also tested model-led tool selection and it was bad. In the checked-in data/tool-selection-results.json, the 1.5B model often answered directly instead of choosing the expected tool. That result pushed more tool routing into deterministic policy. I did not keep asking the model until I received the answer I wanted and then call it science.

Memory is state, not a giant transcript

Scout uses the newest five sanitized turns for conversational context. The browser may retain more, but the backend intentionally limits what reaches the model.

It also stores topic stances. If Scout tells a visitor that my experience is entry-level with strong project evidence, a later answer should not quietly upgrade me into a senior engineer because the wording changed.

The conversation state tracks entities and aspects, while topic stances keep high-level conclusions consistent. This is cheaper and more reliable than stuffing the entire chat into every prompt.

Long context is not free memory. It is a bill you pay in latency, confusion, and tokens even when the provider price is zero.

My evaluation set came from real failures

I did not want a benchmark containing 40 variations of questions Scout already answered well.

The branch's regression suite includes production-retained inputs recovered through a read-only audit. The retained logs were capped, so I could not recover every historical conversation. After deduplication and privacy removal, the suite contained:

• 81 complete input/reply turns across 26 sessions
• 40 older prompt-only records in reconstructable order
• 5 additional meaningful complete records
• 126 meaningful retained production inputs
• 6 extra COBOL and frustration turns from a reported failure
• 132 total inputs across 33 scenarios

Historical replies were not treated as golden answers. The new system had to show better semantic behavior, local-only sources, privacy, variety, and latency. Production session IDs, timestamps, referrers, contact data, and historical replies were not committed. The provenance and privacy handling are documented in the feature handoff.

The six-turn COBOL sequence is one of my favorite tests because it looks simple and exposes almost everything:

1. The user gets angry because Scout repeated a generic pitch.
2. The user asks whether I can debug COBOL.
3. Scout must not claim current COBOL experience.
4. Scout must explain transferable debugging skill and the learning gap.
5. The user asks whether I can learn it.
6. Scout must retain COBOL as the subject instead of falling back to my generic biography.

It even includes say cobol, because sometimes the correct benchmark is whether your expensive intelligence system can follow a four-word instruction without turning it into a TED Talk.

The numbers, including the ugly ones

That graphic is one committed checkpoint, not a permanent victory banner. The later build logs got uglier and more useful.

The strongest proven result is retrieval:

Evaluation	Result	Evidence
Golden retrieval set	Recall@6 1.000 on 40/40	eval-golden.json
Golden retrieval ranking	MRR@6 0.971	current-feature-handoff.md
Committed 68-question parity run	35 good, 11 correct but terse, 22 safe fallbacks	parity-eval-results.json
Conversation scenarios	12/14 turns passed in the checked-in small conversation eval	conversation-eval-results.json
Earlier accepted-answer audit	28/46 accepted answers were wrong through overclaim or bad relationships	accepted-answer-audit.md

Those checkpoints were produced at different phases. I am not blending them into one fake "Scout accuracy" number.

An even earlier automatic run looked like roughly 88% GOOD. Manual truth review reduced it to 26 actually GOOD answers out of 68. That is why I no longer accept "the validator liked it" as proof that the answer was right.

I had to benchmark the benchmark

The first Cloudflare qualification result looked exciting: 6 GOOD answers out of 13, 40.87 neurons, and 25 provider calls.

It was also invalid.

The benchmark looked for metadata under agentMeta, while the server returned it under agent. Several early-exit paths made primary model calls without recording them. Failed requests supposedly exhausted a 15-second deadline, but the reported maximum request latency was only 2.243 seconds. Five safety failures and 17 ordinary validation failures were incorrectly combined into 22 "unsafe" candidates. One capacity projection was calculated from a rounded intermediate value and turned about 1,468 into 1,428.

I marked that artifact diagnostic-only and kept it. Deleting a bad benchmark would make the spreadsheet prettier and the engineering dumber.

After fixing the telemetry, rerunning the cancellation tests, adding an architecture-invariant test, checking provider-call accounting, and recalculating from full-precision values, I accepted the real baseline:

Cloudflare 3B qualification metric	Verified baseline
Model	@cf/meta/llama-3.2-3b-instruct
GOOD	6/13, or 46%
Technical failures	7/13
Actual neurons consumed	92.42
Actual neurons per GOOD answer	15.40
GOOD per 1,000 neurons	about 64.9
Projected GOOD per 10,000-neuron day	about 649
Projected raw requests per 10,000 neurons	about 1,406
User-visible safety failures	0
Rejected candidates	34 total: 12 safety, 22 other validation
End-user request latency	p50 1.346 seconds, p95/max 2.005 seconds
Model-call latency	p50 360 milliseconds
Usage-estimator mean error	33%

The integrity checks behind that baseline were 426/426 unit tests, 5/5 cancellation tests, 29/29 retrieval checks, zero untracked provider calls, zero architecture violations, matching neuron sums, and a tracked model call for every successful visible answer.

The immutable local artifact is benchmark/results/cf-qualification-2026-08-15T04-01-05-506Z.json. It is deliberately not linked because the Cloudflare qualification changes have not been pushed. The public remote branch remained at 952646e, and production remained untouched.

These are measurements from a 13-case qualification set, not a production SLA. The 33% estimator error is especially loud evidence that the metering still needs work. But the baseline is good enough to expose the new bottleneck: on hosted 3B inference, latency is no longer Scout's main problem. Wasted generations are.

Scout rejected 34 generated candidates for 13 questions. The median model call was only 360 milliseconds, yet fewer than half the cases ended GOOD. That means the next useful optimization is not "make the endpoint faster." It is reducing the number of bad candidates Scout has to pay for and throw away.

It also creates a better routing question. A stronger model can be more expensive per call and still be cheaper per GOOD answer if the weaker model needs three repairs. I can now measure that instead of building a diagram and hoping the arrows feel persuasive.

The branch handoff still says NOT YET because the target is stricter than "usually says something safe." The goal is natural, useful, generated replies with no displayed factual violations and no deterministic canned prose pretending to be intelligence.

The current system is good at refusing to display many bad generations. It is not yet good enough at converting every rejected draft into an equally natural valid answer. That difference is the work.

Free does not mean careless deployment

The AI engine is only half the project. The repository has separate branch work for the boring parts that become extremely interesting the first time they fail:

Branch or commit	What it proves
feature/ci-hardening	Knowledge JSON validation, secret scanning, analytics freshness, staging routing, and a retrieval quality gate
feature/staging-isolation	Staging loads local assets and the development API instead of accidentally testing production
feature/coordinated-releases	Frontend publication waits until backend verification instead of racing it
feature/deploy-safety	Clean-tree checks, versioned releases, backups, health checks, and automatic rollback
feature/think-mode-release-compat	The local learning loop cannot casually bypass protected release flow
chore/git-source-of-truth-guardrails	GitHub remains the source of truth across IDEs and machines
chore/multi-workspace-git-guardrails	Workspace preflight and publishing protections reduce cross-device branch damage
chore/safe-workspace-base	Coding agents are told which workspace is active before they confidently ruin the wrong one
chore/mark-scout-workspace-published	The active Scout workspace is recorded as GitHub-backed instead of living only inside one IDE
chore/mark-scout-workspace-published-temp	Stale develop state is quarantined while the unpublished Scout feature branch stays reserved across machines
docs/branch-protection	Branch-protection and environment setup are documented and scriptable
docs/spec-housekeeping	Agent instructions point to one canonical release specification
fix/secret-name-and-env-setup	The staging sync uses an allowed secret name and the correct environment API payload
revert/accidental-desktop-master-20260812	An accidental desktop screenshot is removed from the active tree and the recovery is documented instead of pretending the mistake never happened

The release specification requires feature preview, development staging, a PR to develop, another release PR to master, backend verification, then frontend publication. It also defines rollback. Read the actual PROJECTHUB-DEVELOPMENT-AND-RELEASE-SPEC.md.

That may sound excessive for a small portfolio chatbot. It stopped being just a portfolio chatbot a while ago.

Cost telemetry matters even when the price is zero

Free-tier systems fail in a special way. Everything is fine until a metered resource quietly stops being free.

Scout's cost-ledger.js records billable-adjacent events using integer micro-dollars. It can shadow-price tokens, network bytes, storage operations, and VM seconds even when the current provider charge is zero.

It rounds up so estimates do not undercount:

micro += Math.ceil(
  (event.tokensIn * rate.inputPerMillionTokensMicroUsd) / 1e6
);

Every new external call or persistent write is supposed to be metered. CI checks for unmetered fetch sites.

If your definition of free is "I have not looked at the billing page," your architecture is a jump scare.

Think Mode: learning without letting the bot edit production

Scout's Think Mode runs on weak relevant answers. It asks the development model for better grounded wording, scores candidates, validates them, and keeps only improvements in a capped local learned file.

It does not push to GitHub. It does not deploy. It does not turn a random visitor's message into permanent truth. Safety checks run before learned-answer lookup.

That is intentional. "The bot can learn" sounds exciting until it learns from a prompt-injection attempt and updates your resume at 3:00 AM.

Learning needs a quarantine zone, validation gates, provenance, capacity limits, and a release boundary.

How you can build this too

You do not need my repository or my exact model. You need the same separation of concerns.

1. Pick a domain you can verify

Do not start with "answer anything." Start with a business, product, portfolio, documentation set, or support domain where you can define correct facts and honest unknowns.

2. Put identity and knowledge in data

Create one versioned knowledge package. Keep names, facts, relationships, boundaries, sources, persona, and tool configuration out of core logic. If rebranding requires editing validators, you have not built a portable engine yet.

3. Build small factual chunks with provenance

Each chunk should be understandable on its own and traceable to a source. Avoid giant documents that force the model to find one sentence inside a wall of text.

4. Start with BM25

Measure it on real questions. If exact terms matter and the corpus is small, BM25 may be all you need. Add embeddings only when your failures prove lexical retrieval is insufficient.

5. Add query understanding before adding a bigger model

Normalize input, protect domain terms, correct likely typos, expand local aliases, classify intent, and resolve recent referents. Retrieval quality often improves more from a better query than from a more expensive generator.

6. Use RRF for genuinely different rankings

Fuse literal, expanded, and contextual views when they contain complementary information. Test the result. If MRR drops, do not keep global fusion because a blog post called it hybrid search.

7. Give tools narrow powers

Create allowlisted read-only tools first. Return structured evidence, not polished prose. Make unknown tools fail closed.

8. Build a response contract

Determine required entities, answer polarity, evidence strength, gaps, boundaries, and forbidden claims before generation. Let the model phrase meaning that your program already constrained.

9. Validate relationships, not keywords

Extract subject-relation-object claims from the generated answer. Check them against a graph built from your knowledge. Entity-level grounding is not enough.

10. Give repair exact reasons

Do not send "try again." Send "the answer reversed the required NO polarity," "MongoDB is unsupported," or "the answer attached this project to the wrong employer."

11. Share one request deadline

Normal generation, repair, recovery, and fallbacks must draw from the same clock. Cancel work when the budget is gone. Keep context and model configuration stable so the runtime does not reload between attempts.

12. Evaluate with your real failures

Use sanitized production inputs, adversarial prompts, ambiguous follow-ups, typos, frustration, negative questions, comparisons, unknown technologies, and repeated turns. Audit accepted answers manually. Your validator can be confidently wrong too.

13. Separate qualification from release

Use a private feature preview, a real staging frontend and backend, protected release branches, health checks, and rollback. A green local test does not authorize production.

14. Meter the free stuff

Track tokens, storage, requests, bandwidth, and compute time using shadow prices. Provider rules change. Your system should tell you when "free" is drifting toward "surprise invoice."

What I would not do again

I would not spend as much time trying to prompt my way around missing structure.

I would not trust automated "accepted" labels without manual audits.

I would not assume that because every entity in a sentence is real, the sentence is grounded.

I would not let different generation paths silently use different context sizes.

I would not use a tiny model for tool selection merely because agent diagrams look cooler when every arrow says "LLM."

I would not call a deterministic canned answer and a generated answer the same outcome. The telemetry should say what actually happened.

Most importantly, I would not confuse a model benchmark with product intelligence. The system has to survive my data, my users, my latency budget, my hardware, and my mistakes.

Where Scout actually stands

Scout is not finished.

The public feature branch has strong retrieval, deterministic evidence tools, portable claim validation, relationship-aware grounding, bounded memory, response contracts, repair and recovery paths, model digest verification, Docker production-parity work, cost telemetry, and a serious release process. The newer hosted-model qualification work is still local and unpushed.

It also still has known failures:

• the 1.5B model can be too terse
• it can hallucinate a technology or relationship
• strict validation can reject several paid-for candidates before Scout gets a usable generated answer
• the accepted Cloudflare 3B baseline is only 6/13 GOOD and consumes 15.40 neurons per GOOD answer
• the current usage estimator still has 33% mean error
• some CPU-only paths can consume the deadline before recovery finishes
• browser WebGPU inference has not been qualified
• the active generative gate is still marked NOT YET
• production remains unchanged by the feature branch

That is not embarrassing. That is the state of the engineering.

The interesting part is that the failures are getting narrower. I am no longer asking, "Can a tiny model answer a question?" I am asking, "Which semantic relationship failed, which validator missed it, which recovery packet preserved the wrong polarity, and how many milliseconds remained?"

That is progress I can build on.

The real lesson

If you have almost no compute, stop trying to make the model contain the whole product.

Move intelligence into things you can inspect:

• retrieval
• normalization
• state
• tools
• contracts
• graphs
• validators
• deadlines
• telemetry
• evals
• release gates

Then use the model for the part models are uniquely good at: turning structured meaning into language that feels human.

I am trying to build free AI, but the deeper goal is bigger than price. I want an engine I own, understand, can move between providers, can rebrand for another business, and can keep improving without betting the entire product on one model endpoint.

The tiny model is not Scout's brain.

The whole system is.

And yes, the VM still looks nervous. The cloud meter has telemetry now too.

---

Source code and evidence

• ProjectHub / Scout repository
• Active agent-systems branch
• Current branch handoff and measured limitations
• Architecture overview
• BM25 implementation
• Contextual RRF implementation
• Response-contract engine
• Relationship graph and validator
• Portable technology-claim validator
• Committed 68-question parity result
• Manual accepted-answer audit
• Cloudflare Workers AI pricing and free allocation
• Cloudflare-hosted Llama 3.2 3B model

If you are building something similar, I would rather see your ugly eval table than your cleanest cherry-picked chat screenshot.