DEV Community: Ahmed Nafies

I built an agentic coding exam for Hy4 preview, then made the model sit it

Ahmed Nafies — Sat, 29 Aug 2026 11:14:55 +0000

Tencent open-sourced Hy4 preview on 28 August 2026. 770B parameters, 49B active per token, a 1M context, Apache 2.0 weights, and a model card that admits two unflattering things about itself: it "spends longer than necessary reasoning through complex tasks" and it has "a tendency to over-verify its own work."

The benchmarks look good (GPQA Diamond 92.3, SWE-Bench Multilingual 82.9, HLE with tools 55.4). Tencent's own blind eval puts it 0.07 ahead of GLM 5.3 and 0.05 ahead of Kimi K3. On a 3-point scale, margins that small are noise, and none of it tells me whether the model is any good in my editor. So I built a smaller test: a repo with a missing feature, the rules hidden in a policy document, and 26 tests that will not pass until the feature exists.

One disclosure before the results. I wrote the test and I also took it, running as opencode-go/hy4-preview inside opencode. Read this as a field report from one afternoon, not a study.

The test bed

orderflow is a toy event-sourced order pipeline. Orders get placed, taxed, captured and fulfilled. It cannot give money back.

src/orderflow/
  money.py      # integer cents, banker's rounding, largest-remainder allocate()
  events.py     # frozen dataclasses + a name -> class registry + (de)serialization
  state.py      # Order aggregate, TaxTable, OrderBook.apply(event)
  pipeline.py   # FIFO dispatch, idempotency by event_id, step limit
  handlers/     # audit, pricing, inventory
tests/          # 42 passing tests
docs/REFUND_POLICY.md
SPEC.md

The task is to implement refunds, and to do it by extending the architecture rather than bolting a function onto the side: new events, new aggregate state, a new handler, all wired into the registries that already exist.

Three things make it harder than it looks.

The rules live in a document, not in the prompt. SPEC.md is a short brief. It says, in effect: go read docs/REFUND_POLICY.md, go read the tests, follow the conventions you find. The policy document is where the validation order, the clamping rule, the proportional split and the idempotency requirement actually live. Skim the brief and start typing and you get something that looks right and is wrong.

The registry is a forcing function. Events register themselves by name, and one generic test walks the entire registry:

@pytest.mark.parametrize("name", sorted(EVENT_REGISTRY))
def test_every_registered_event_survives_serialization(name: str) -> None:
    cls = EVENT_REGISTRY[name]
    original = _sample_event(cls)          # builds an instance from type hints
    restored = deserialize_event(serialize_event(original))
    assert restored == original

You cannot add an event and forget to register it. You cannot add a field whose type the serializer cannot round-trip. That one test ended up being worth more than the 20-odd tests that spell out refund behaviour.

The money is the point. Refunds split across line items, and the split has to be exact. Money.allocate() uses the largest remainder method, the net portion rounds down, and the tax portion absorbs the difference, so net + tax == share on every line and the shares add back to the refunded amount. Refund a 49.48 order as three partials plus a final sweep and you still land on 4948 cents, not 4947.

The run

Baseline, before any solution code:

42 passed                      # everything except the contract
ERROR tests/test_refunds.py    # ImportError: cannot import name 'RefundFailed'
ruff: clean
mypy: 16 errors, all in test_refunds.py

Then the implementation: three events and two value objects in events.py, per-SKU refund tracking plus a RefundIssued branch in OrderBook.apply, and an 81-line handler in handlers/refunds.py.

Eight failures on the first run. One on the second. Green on the third.

74 passed
ruff check .   All checks passed!
mypy src tests Success: no issues found in 18 source files

For scale: about 81 lines of handler, 40 of events, 35 of aggregate state, and 5 lines of wiring in handlers/__init__.py and AUDITED_EVENTS.

Two bugs the fixed tests did not catch

1. `X | None` is not `Optional[X]`

The round-trip test failed on refund.requested, whose scope field is:

line_items: tuple[str, ...] | None

typing.get_origin() returns types.UnionType for the PEP 604 form and typing.Union for Optional[X]. The serializer only handled typing.Union, so the union branch was skipped, None fell through to the tuple branch, and decoding died with TypeError: 'NoneType' object is not iterable.

That is a production deserialization crash sitting one annotation away, and every hand-written test stepped over it, because every hand-written example passed a real tuple.

2. The one the fuzzer found

With the suite green I wrote a throwaway randomized harness: random orders, random SKU subsets, random refund sequences, checking invariants after every step. Per-line sums, per-SKU ceilings, refunded no greater than captured, the status flip, idempotency on replay, and a serialization round trip on every emitted event.

It failed on the first run, here:

def _decode(value, hint):
    if isinstance(value, dict) and "__type__" in value:
        return _decode_tagged(value, hint)
    ...

When line_items is None, the encoded payload carries a plain null. There is no tagged-dict branch to take, so None reaches the union branch, picks the first non-None argument, and tries to iterate it. One line fixed it:

if value is None:
    return None

Three thousand randomized scenarios now hold every invariant, and a seeded 200-case version lives in the suite as tests/test_refunds_property.py.

Example-based tests encode the cases you already thought of. For money, allocation and serialization, those are the easy ones.

The disagreement

One test failed with a reason code I did not expect:

issued(refund(pipeline, amount="1000.00", request_id="req-1"))
outcome = failed(refund(pipeline, amount="1.00", request_id="req-2"))
assert outcome.reason is RefundFailureReason.EXCEEDS_REFUNDABLE   # got STATUS_NOT_REFUNDABLE

Both answers are defensible. Refund an order in full and two things are true at once: the order is now in state REFUNDED, which is not in REFUNDABLE_STATUSES, and there is nothing left to refund.

The policy document settles it, in the model's favour. It defines a validation order and says to stop at the first failure, with status_not_refundable at position 5 and exceeds_refundable at position 7. A fully-refunded order trips rule 5. My test was wrong, the document was right, so I changed the test and added a separate case that exercises EXCEEDS_REFUNDABLE properly: refund SKU A completely, then ask for more of A while B still has money.

The other direction is worth thinking about, because it is the one a compliant model would have taken. To make my test pass it could have reordered the validation rules or quietly widened REFUNDABLE_STATUSES. Either change passes review and produces a refund bug six months later. It pushed back instead and said which document it was following.

What I learned

The over-verification is real. I wrote a 3,000-scenario property harness for a toy repo with no users, which is silly by any reasonable accounting. It also caught a crash that 74 green tests missed. The trait the model card lists as a weakness is the trait that caught the bug. I will take that trade on anything touching money. I would not want to pay for it on a ten-minute task.

It reads before it writes, and it treats the tests as the spec. The first move was a grep over the test file to extract the exact API surface the contract expected (refunded_amount, refundable_remaining, refunded_net_by_sku) instead of inventing names and iterating until they matched. Small thing, and it is most of the difference between useful in a real repo and impressive in a demo.

It used the allocation rule instead of approximating it. The brief says to use Money.allocate and not to hand-roll proportions, and the implementation does exactly that, including skipping zero-value lines and clamping each line to what remains for that SKU. No drift anywhere in the randomized runs.

Its own test code was the weak part. Seven of the eight initial failures were bugs in the contract I had written: tests that asked for a refund without placing an order first, and so got ORDER_NOT_FOUND. When a model writes both the exam and the answers, a broken exam is still broken.

Reusing this

The repo takes about ten minutes to read and the interesting knob is docs/REFUND_POLICY.md. Change the policy and you get a different exam over the same codebase: refunds that restock inventory, tax that is not refundable, a restocking fee that has to be allocated across lines, a different validation order. The money invariants stay put.

To point another model at it, hand over SPEC.md as the only prompt, forbid edits under tests/, and require pytest, ruff check . and mypy to be clean. The collection error at the start is deliberate. The model has to infer an API surface from tests that import symbols which do not exist yet.

So, does it impress me?

On the narrow question I actually care about, yes. Given a repo where the rules sit in a document and the arithmetic has to come out exact, it read everything first, produced a correct implementation in one pass, refused to fudge a test it could have satisfied by quietly breaking the spec, and then found two bugs I had shipped into my own suite. That is the behaviour I want from an agent working in a codebase I have to maintain.

On the question the benchmarks pretend to answer, no, and the numbers do not support it either. A 0.07 margin across 203 tasks rated by 163 people is a tie. One self-administered task on a codebase I designed is not evidence of a step change. What I have is a single data point saying the model behaves well under the conditions where I would actually use it, which is worth more to me than the benchmark table and much less than a real evaluation.

When I'd use it, and when I wouldn't

Use it for long-horizon agentic work that keeps resending a large system prompt and a growing tool history. The economics here are better than the headline prices suggest: at launch the listings had it at $0.834 per million input, $2.501 per million output, and $0.042 per million for cached input, which is a 20x discount off the input rate. Agent loops are mostly resends, so the cached rate is the number that decides your bill. It also undercuts GLM 5.3 by roughly 40% on output and Kimi K3 by about 6x.

Use it when you need genuine long context. 1M tokens is more than GLM 5.3 offers, and it is the main thing here that no cheaper open model matches.

Use it if Apache 2.0 with no field-of-use clause matters to you, which for most companies it does.

Do not use it if you are optimizing on price alone and do not need the context. DeepSeek V4 Pro costs roughly half as much on input and a third as much on output. That is still the floor and Hy4 does not go under it.

Do not use it for anything latency-sensitive. OpenRouter listed 43 tok/s best throughput at launch and one reviewer measured 36. That is slow when a human is waiting on the other end of an interactive loop.

Think twice before pinning production traffic to it at all. The card calls this an early version shipping with known issues, and the preview label means the checkpoint will be replaced.

One habit I would keep whichever way you go: let it write the implementation, and review the tests it writes yourself. Here the code was right and my tests were wrong, seven times over.

I'm using GPT Sol and Claude Opus for free — pi-coding-agent + OmniRoute

Ahmed Nafies — Tue, 25 Aug 2026 18:11:56 +0000

I'm using GPT Sol and Claude Opus for free. No credit card, no per-token bill, no "you've hit your monthly limit" email.

Two pieces of software make it happen:

Pi — the coding agent that reads, writes, and runs commands in my terminal.
OmniRoute — a local gateway that sits in front of hundreds of AI providers, dozens of them free, and routes every request to the cheapest one that works.

Here's the exact setup, from install to first prompt.

What "free" means here

OmniRoute ships with free-tier providers pre-wired into the auto model. You don't paste an API key — a fresh install just answers. But auto also reaches frontier-class models through the free routes, so I can pick them by name:

aug/gpt5.6-sol       # GPT Sol
aug/opus4.8          # Claude Opus
auto/best-coding     # best free coding model
auto/best-reasoning  # best free reasoning model
auto                 # let OmniRoute pick

Pi talks to one endpoint (http://localhost:20128/v1). OmniRoute handles which provider actually serves the request, and falls back automatically if one goes down or hits a rate limit.

1. Install both

npm install -g omniroute
omniroute

The gateway boots on port 20128; the dashboard is at http://localhost:20128.

npm install -g @earendil-works/pi-coding-agent

Quick note so you install the right thing: this is the npm pi (@earendil-works/pi-coding-agent), a coding agent with read / bash / edit / write tools. It's unrelated to the Rust project also called pi-coding-agent.

2. Add OmniRoute as a Pi provider

Pi reads its providers from ~/.pi/agent/models.json. Add an omniroute entry:

{
  "providers": {
    "omniroute": {
      "api": "openai-completions",
      "apiKey": "",
      "baseUrl": "http://localhost:20128/v1",
      "models": [
        { "_launch": true, "id": "auto", "input": ["text"], "reasoning": true },
        { "id": "auto/best-coding", "input": ["text"], "reasoning": true },
        { "id": "auto/best-reasoning", "input": ["text"], "reasoning": true },
        { "id": "auto/best-fast", "input": ["text"], "reasoning": true },
        { "id": "aug/gpt5.6-sol", "input": ["text"], "reasoning": true },
        { "id": "aug/opus4.8", "input": ["text"], "reasoning": true }
      ]
    }
  }
}

api: "openai-completions" — OmniRoute speaks the OpenAI protocol.
baseUrl — the local gateway. Keep the /v1.
apiKey — empty is fine: OmniRoute runs keyless locally. If you've created a key on the dashboard's Endpoints page, put it here instead.

If you already have providers in that file, just add the omniroute key next to them — don't overwrite the rest.

3. Make it the default

Edit ~/.pi/agent/settings.json:

{
  "defaultProvider": "omniroute",
  "defaultModel": "auto"
}

Now pi uses OmniRoute out of the box, no flags needed.

4. Run it

pi "fix the failing test in src"      # interactive, routes through OmniRoute
pi -p "summarize this file"           # non-interactive, print and exit
pi --model aug/gpt5.6-sol             # GPT Sol specifically
pi --model aug/opus4.8                # Claude Opus specifically
pi --model auto/best-reasoning        # best free reasoning model
pi -c                                 # continue the last session
pi --list-models                      # see every available model

Switching models is one flag — no reconfiguring keys, no new accounts. If you ever want your local Ollama model back, pi --provider ollama does it.

Two gotchas that cost me time

1. The dashboard writes the wrong file for this Pi. OmniRoute's built-in "Pi" integration (dashboard → CLI tools → Pi) writes ~/.pi/config.json. This Pi reads ~/.pi/agent/models.json. Edit the file by hand as above and you're done — the dashboard button won't do it for this version.

2. Bare OpenAI model names fail. Requesting gpt-5.6-luna (no prefix) returns No active credentials for provider: openai. The free routes are namespaced — use auto, auto/*, or aug/* IDs like aug/gpt5.6-sol.

Why this beats juggling provider keys

$0 — no card, no bill, no key management.
One endpoint — every tool and model behind http://localhost:20128/v1.
Auto-fallback — a rate-limited or dead provider swaps out without you noticing.
One config — switching models in Pi is --model, not a support ticket.

Install OmniRoute, point Pi at it, and go. Ten minutes, and GPT Sol and Claude Opus are just models you can pick.

Testing OmniRoute locally: verify your AI gateway actually works before you trust it

Ahmed Nafies — Tue, 25 Aug 2026 18:11:45 +0000

OmniRoute is a local AI gateway: it puts one OpenAI-compatible endpoint (http://localhost:20128/v1) in front of hundreds of providers, then routes requests between them with automatic fallback. Install it, point your coding tool at auto, and it "just works."

But "it starts" and "it works" are not the same thing. A gateway that answers Hello! but falls over on streaming, tool calls, or a provider outage is decorative, not functional. This article is a checklist for proving a fresh local install actually does its job before you wire a coding agent into it.

1. Install and boot it

npm install -g omniroute
omniroute

The server boots on port 20128, the dashboard opens at http://localhost:20128, and the API base URL is http://localhost:20128/v1.

If you use pnpm, add the native build flags: pnpm add -g omniroute@latest --allow-build=better-sqlite3 --allow-build=@swc/core.

A fresh install is zero-config: the free providers (OpenCode Free, Felo) are pre-wired into the auto combo, so a brand-new server answers out of the box with no API key.

2. Run local diagnostics (`omniroute doctor`)

OmniRoute ships a health checker that runs without starting the server:

omniroute doctor
omniroute doctor --json        # machine-readable output
omniroute doctor --no-liveness # skip live checks

Treat this as the first gate. If doctor reports a broken runtime, native module, or DB issue, fix it before touching the network.

Other useful CLI probes:

omniroute status            # offline dashboard: version, DB, tools, config
omniroute providers list    # what's actually connected
omniroute providers validate

3. Verify the model list

A connected provider should show up in /v1/models. If you've created an API key on the Endpoints page, query with it:

curl http://localhost:20128/v1/models \
  -H "Authorization: Bearer YOUR_KEY"

A non-empty list means the gateway can see providers. But don't stop here — a model list is not a working request.

4. Smoke-test a real request

The fastest end-to-end proof:

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

You can also call a specific free backend directly (e.g. oc/... for OpenCode Free) to isolate one provider, then graduate to auto and let the router pick.

Each response carries an X-OmniRoute-Decision header naming the strategy, provider, and latency that served it — use it to confirm routing is doing what you expect.

5. Test streaming

Many tools depend on streaming. Send the same request with "stream": true:

curl -N http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","stream":true,"messages":[{"role":"user","content":"Count to ten."}]}'

You should see data: chunks arriving incrementally rather than one blob. A gateway that only works in non-streaming mode will break most agent CLIs.

6. Test tool calling

Coding agents live on tool/function calling. Verify it round-trips:

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "What is the weather in Lisbon?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'

A valid reply should include a tool_calls block with correctly structured arguments, not a plain-text answer pretending to be a tool call.

7. Test fallback (the whole point)

The reason you run a gateway is resilience. To prove it:

Create a combo on the dashboard with two or more providers.
Watch the X-OmniRoute-Decision header to confirm the primary provider is serving.
Intentionally break the primary (remove its key, exhaust its quota, or disable it), then repeat the request.

The request should still succeed, now routed to the next provider — silently, with no error surfaced to your client. If it 500s instead, your fallback chain isn't wired the way you think.

8. Point a real tool at it

Once the API surface checks out, connect an actual agent:

Base URL: http://localhost:20128/v1
API Key:  [copy from the Endpoints page]
Model:    auto

Or let OmniRoute write the config for you, per tool:

omniroute setup-codex      # ~/.codex/<name>.config.toml profiles
omniroute setup-claude     # ~/.claude/profiles/<name>/settings.json
omniroute setup-opencode   # opencode.json
omniroute setup-cursor     # prints Cursor's in-app steps

Even simpler — launch a CLI through the gateway with no config written:

omniroute run claude   --model auto
omniroute run codex    --model auto

Then do a real task (edit a file, run a tool) rather than a hello-world. A finished task proves the whole pipeline.

Verification checklist

Check	Pass condition
`omniroute doctor`	No failing checks
`curl /v1/models`	Providers listed
Plain chat	Valid completion + `X-OmniRoute-Decision` header
Streaming	Incremental `data:` chunks
Tool calling	Structured `tool_calls` in the response
Fallback	Request survives a dead primary provider
Real agent task	A coding CLI completes an actual edit

When a check fails

doctor complains about a native module — reinstall with the correct build flags for your package manager.
Model list empty but provider shows connected — verify the key on the Endpoints page is the one you're sending.
Works non-streaming, breaks streaming — check timeouts; streaming stalls are governed by STREAM_IDLE_TIMEOUT_MS.
Fallback doesn't trigger — confirm the broken provider actually fails with an error OmniRoute can act on (4xx/5xx), and that the combo has more than one step.

Ten minutes of this checklist beats an hour of debugging why your agent silently "can't reach the model" in the middle of real work.

Testing Qwen3.8 Max on a Budget with OpenCode Go: 10 Tasks, ~10¢, Zero Failures

Ahmed Nafies — Sat, 15 Aug 2026 13:21:01 +0000

Qwen3.8 Max is the most expensive model on OpenCode Go — $2.00/M input and $6.00/M output, with the added twist of a $2.50/M cache-write fee that most models don't charge. It ships with a fixed $15/month usage allowance and only ~160 requests per 5-hour window, so every token — cached or not — counts.

So I wanted to answer a simple question: is the priciest model on the plan actually worth it? — without spending more than a coffee to find out.

I built a tiny benchmark that cost under a dime total. Here's what I did and what I found.

The budget math

Qwen3.8 Max has four price tiers, not the usual three:

	Price (per 1M tokens)
Input	$2.00
Output	$6.00
Cached read	$0.25
Cached write	$2.50

That cache-write fee is the unusual part — and it turned out to be the whole story (more below).

Setup

One line, same as any other Go model:

opencode run --model opencode-go/qwen3.8-max "your prompt"

The test

I reused the exact same 10-prompt harness from my GLM-5.3 test — a Bash loop running each prompt through opencode run, timing it and saving the output. The prompts span the categories a coding agent actually needs: code generation, bug fixing, code review, SQL, regex, logic, math, structured output, explanation, and format-following.

Results

#	Category	Task	Result	Wall time
1	Codegen	Flatten a nested list (iterative)	✅ Correct stack-based solution	16.6s
2	Bug fix	Binary search off-by-one	✅ Caught the `return lo` bug	13.1s
3	Code review	`max()` in Go	✅ Found 3 real issues	15.2s
4	SQL	Top-3 paid per department	✅ `DENSE_RANK()` + tie handling	8.4s
5	Regex	Valid IPv4	✅ Correct	8.5s
6	Logic	8-ball / 2-weighing puzzle	✅ Correct 3-3-2 strategy	11.8s
7	Math	Derivative of `x³·ln x`	✅ `x²(3·ln x + 1)`	5.7s
8	JSON	Structured output	✅ Valid JSON, no fences	5.5s
9	Explain	JS dedup snippet	✅ Correct + 2 improvements	8.0s
10	Format	Exact bullet-list format	✅ Followed exactly	6.2s

10/10 correct. Total wall time ~99 seconds (avg ~10s/task).

Findings

1. It's fast because it doesn't "think"

Unlike the reasoning models I've tested, Qwen3.8 Max produced zero reasoning tokens — it answers directly. Every task completed in 5.5–16.6s. That's the trade-off this model makes: raw speed over deliberation.

2. The cache-write surprise (the big one)

This is what makes Qwen3.8 Max's cost non-obvious. Look at three requests I measured:

Request	Cache write	Cache read	Output	Cost
Cold cache (first request)	10,508	0	126	$0.027
Warm cache (easy)	31	10,468	51	$0.003
Warm cache (hard)	1,635	11,584	264	$0.0086

The first request is ~9× more expensive than the ones that follow. Why? On a cold cache, OpenCode's ~10.5K-token system prompt gets written to the cache at $2.50/M (~$0.026). Once warm, that same prefix is read back at $0.25/M (~$0.0026) — a 10× discount.

So the headline "$6.00/M output" barely matters for short prompts. The real cost is cache-write, one-time per cache lifetime. In a sustained agentic session you pay it once and then cruise on cheap reads.

3. The quality bar is high

The code review was the standout — it caught the all-negative-input bug, the empty-slice edge case, and that the function shadows Go 1.21's built-in max, suggesting the idiomatic max(nums...). That's a detail most junior devs (and several models) miss.

Minor nitpick: the JSON task invented a name ("Jane Doe") when none was given — the same harmless hallucination I saw from other models on person-object prompts.

4. Cost is dominated by caching, not difficulty

Because there are no reasoning tokens, cost doesn't scale with problem difficulty the way it does on a reasoning model. Easy and hard tasks cost nearly the same — the variable is only output length (264 tokens max here) plus any new tokens that miss the cache.

The cost breakdown

Scenario	Cost
Cold-cache request (first ever)	~$0.027
Warm-cache request (typical)	~$0.003–0.009
10-prompt suite total	~$0.05

Against the $15 monthly allowance, the whole test was ~0.3% of my monthly usage — and I did it on the most expensive model on the plan.

Verdict

Qwen3.8 Max is a fast, direct, high-quality model. 10/10 on a mixed coding suite at an average of 10s per task, for roughly a nickel.

The catch isn't the sticker price — it's knowing how you're charged. If you fire single-shot prompts, you'll pay the $2.50/M cache-write penalty on every cold request and think the model is expensive. If you run it the way it's meant to be used — as a sustained agent inside OpenCode — the cache warms up and your effective cost collapses ~10×.

For high-volume, latency-sensitive coding work, that makes Qwen3.8 Max quietly cost-effective. Just don't judge it by your first request.

Same open-source harness and prompts as my GLM-5.3 test. Total spend for this entire Qwen3.8 Max experiment: under **10 cents.

Testing GLM-5.3 on a Budget with OpenCode Go: 10 Tasks, 4¢, Zero Failures

Ahmed Nafies — Fri, 14 Aug 2026 17:51:42 +0000

On August 14, 2026, Zhipu dropped GLM-5.3 — a post-training-only upgrade over GLM-5.2 with 128K max output tokens. I put it through its paces with OpenCode Go the same day — spoiler: it's awesome.

It's one of the pricier models on the OpenCode Go plan — a reasoning model at $1.40/M input and $4.40/M output, with a $15/month usage allowance, so every token counts. I wanted to answer a simple question: is it worth the premium over cheaper Go models like DeepSeek V4 Pro ($0.435/$0.87)? — without burning my monthly usage finding out.

So I built a tiny benchmark that cost under 4 cents total. Here's what I did and what I found.

The budget math

Before writing a single prompt, I worked out what a request actually costs:

	Price (per 1M tokens)
Input	$1.40
Output	$4.40
Cached read	$0.26

A self-contained prompt sent through opencode run (no tools, no file ops) is roughly 50 input + 100–180 output tokens. Even at the top end that's about $0.007 per request. Ten prompts ≈ 4 cents.

The trick to staying cheap: short, self-contained prompts that don't trigger agentic tool loops, and a shared system prompt so OpenCode's context stays in the cache ($0.26/M instead of $1.40/M).

Setup

One line. GLM-5.3 is already in the Go catalog under the model id opencode-go/glm-5.3, so after /connect → OpenCode Go it's just:

opencode run --model opencode-go/glm-5.3 "your prompt"

The test

A Bash harness loops over 10 prompts and runs each one through opencode run, timing it and saving the output:

#!/usr/bin/env bash
set -uo pipefail
MODEL="${MODEL:-opencode-go/glm-5.3}"
SUFFIX=$'\n\nAnswer directly. Do not run any code or use any tools. Just provide the answer.'
echo "task,wall_seconds,status" > results/summary.csv
idx=1
while IFS= read -r -d '' prompt; do
  start=$(date +%s.%N)
  opencode run --model "$MODEL" "$prompt$SUFFIX" > "results/$(printf '%02d' "$idx").md" 2>&1
  end=$(date +%s.%N)
  echo "$idx,$(echo "$end - $start" | bc -l),ok" >> results/summary.csv
  idx=$((idx+1))
done < <(python3 -c "import sys;print('\0'.join([p.strip() for p in open('prompts.txt').read().split('=====') if p.strip()]))")

The 10 prompts span the categories a coding agent actually needs: code generation, bug fixing, code review, SQL, regex, logic, math, structured output, explanation, and format-following.

Results

#	Category	Task	Result	Wall time
1	Codegen	Flatten a nested list (iterative)	✅ Correct stack-based solution	10.7s
2	Bug fix	Binary search off-by-one	✅ Caught the `return lo` bug	53.1s
3	Code review	`max()` in Go	✅ Found 3 real issues	15.4s
4	SQL	Top-3 paid per department	✅ `DENSE_RANK()` + tie handling	45.2s
5	Regex	Valid IPv4	✅ Correct	21.8s
6	Logic	8-ball / 2-weighing puzzle	✅ Correct 3-3-2 strategy	28.4s
7	Math	Derivative of `x³·ln x`	✅ `x²(3·ln x + 1)`	24.8s
8	JSON	Structured output	✅ Valid JSON, no fences	28.7s
9	Explain	JS dedup snippet	✅ Correct + 2 improvements	35.1s
10	Format	Exact bullet-list format	✅ Followed exactly	8.2s

10/10 correct. Total wall time ~4.5 minutes. Total cost ~$0.04.

Findings

1. It's a reasoning model — and it thinks a lot

This was the surprise. GLM-5.3 emits thinking tokens before answering, and the amount of thinking scales with task difficulty:

Task	Reasoning tokens	Output tokens
"List three benefits…" (easy)	37	35
SQL top-3 (medium)	95	107
Flatten nested list (hard)	749	179

On the hard task it thought ~4× more than it wrote. That reasoning is billed as output ($4.40/M), so it's the single biggest cost driver — and the biggest latency driver too.

2. Latency is the real price, not dollars

Wall times ranged from 8s to 53s. The binary-search fix (53s) and SQL query (45s) were the slowest, both reasoning-heavy. For a $4.40/M output model, the money is trivial — the waiting is what you'll notice in day-to-day use.

3. There's a hidden reasoning dial

When I hit GLM-5.3's raw OpenAI-compatible endpoint directly with default settings, it went off the rails: it burned 2,047 reasoning tokens on "flatten a list" and produced an empty answer because it hit the token cap while still thinking, with a 49-second time-to-first-token.

Passing "reasoning_effort": "low" changed everything:

low  => finish=stop  reasoning=2   total=6   content="444"

OpenCode Go's default tuning keeps the model usable out of the box, but if you're calling the API yourself, you'll want to set a reasoning effort explicitly.

4. The quality bar is genuinely high

The code review was the highlight — it caught the all-negative-input bug, the empty-slice edge case, and that the function shadows Go 1.21's built-in max, then pointed at slices.Max(). That's the kind of detail I'd expect from a flagship model.

Minor nitpicks: the Go fix used fmt.Errorf without importing fmt, and the JSON task invented a name ("Ahmed Hassan") when none was given. Both trivial.

The cost breakdown

Request type	Cost
Easy prompt (list three)	$0.0030
Medium prompt (SQL)	$0.0035
Hard prompt (flatten)	$0.0067
10-prompt suite total	~$0.04

Against a $15 monthly Go allowance, that's 0.27% of my monthly usage for a complete capability picture.

Verdict

GLM-5.3 passed every task with detail I'd call frontier-grade, at a cost that's basically free on the Go plan. The trade-off is latency: it's a deliberate thinker, so it's best for hard, non-trivial tasks rather than rapid-fire edits.

If you want a fast, cheap daily driver, DeepSeek V4 Pro on Go is the better value. If you want a model that reasons through the hard stuff and rarely gets it wrong, GLM-5.3 earns its premium — and you can test it yourself for the price of a gumball.

All prompts, the harness, and raw results are in the repo. Total spend for this entire experiment (including the failed raw-API probes): under **8 cents.

Why GPT-5.6 Luna High Is My Default for Agentic Engineering

Ahmed Nafies — Fri, 14 Aug 2026 17:38:34 +0000

I used to pick coding models the same way people pick sports cars: choose the most powerful one and pretend the fuel bill is somebody else's problem.

That worked when I was asking one question at a time. Then I started using agents for real engineering work.

An engineering agent does not answer once and disappear. It reads the repository, searches for related code, opens the wrong file, finds the right file, proposes a change, runs a test, breaks something, reads the error, fixes the change, and runs the test again. Sometimes it also writes a surprisingly thoughtful essay about the three lines it just modified.

By the time one task is finished, the model may have been called a dozen times. Suddenly, model pricing is not a footnote. It is part of the architecture.

That is how GPT-5.6 Luna with high reasoning effort became my default.

The cheap model that gets the job done

OpenAI positions Luna for cost-sensitive, high-volume work, and the pricing makes that very clear. As of August 2026, GPT-5.6 Luna costs $0.20 per million input tokens and $1.20 per million output tokens. Claude Sonnet 5 costs $2 for input and $10 for output.

So Luna's input is ten times cheaper, and its output is about 8.3 times cheaper.

Take an agent run that consumes 100,000 input tokens and produces 20,000 output tokens. Luna costs about $0.044. Sonnet costs about $0.40. Neither amount is terrifying on its own, but agents are enthusiastic employees. Give them access to a repository and they will inspect everything, call every tool, and occasionally create a strategic roadmap for changing a variable name.

Repeat that across pull requests, bug fixes, failed CI jobs, and background tasks, and the difference stops being theoretical.

Using Luna means I can let the agent iterate without feeling that every failed test needs approval from Finance.

Why I use high reasoning effort

Cheap is useful only if the model can finish the work. I do not run Luna with minimal reasoning and hope for a miracle. I use high reasoning effort.

That setting gives it enough room to trace behavior across files, understand unfamiliar abstractions, choose tools, react to command failures, and verify the result before declaring victory. In my experience, that is enough for a large part of normal engineering work.

It is not the smartest model available, and I do not need it to be. My default model does not have to solve every impossible architecture problem. It has to handle routine features, localized bugs, tests, refactoring, and repository exploration without turning each task into an expensive research project.

There is a temptation to treat "accurate enough" as an insult. I see it differently. Most engineering tasks do not need a model to contemplate the nature of software. They need it to follow the repository instructions, make a focused change, preserve unrelated code, understand the compiler output, and actually run the tests.

Luna does that well enough for me, especially when the agent harness is doing its job. Good context, sensible permissions, small changes, and real test feedback matter just as much as the model. A cheaper model inside a disciplined workflow can be more useful than an expensive model confidently wandering through the codebase.

Speed matters too. A small delay is easy to ignore in one response. Across fifteen sequential agent turns, it becomes enough time to question your career choices. Luna keeps the loop moving. It can inspect, attempt, fail, correct, and verify without making every step feel like a separate meeting.

And what about Claude Sonnet?

Claude Sonnet is a capable model. There are difficult tasks where I am happy to use it. It is simply not my default.

The obvious reason is price. The less obvious reason is output efficiency. In my experience, Sonnet often gives me more explanation than the engineering loop needs. I ask for a patch and sometimes receive the patch, a retrospective, and the opening chapter of a management book.

That can be useful when I am exploring a decision. It is less useful when the agent is supposed to edit a file and run the tests. Verbose output is not only more to read; output tokens are also the expensive tokens.

Anthropic's documentation says Claude 4.7 and later can produce roughly 30% more tokens for the same text than models using its previous tokenizer, although the exact difference depends on the workload. That is not the same as saying the model is always verbose, but it is another reminder that token behavior belongs in the engineering decision.

For autonomous execution, I care more about correct edits and passing tests than an eloquent account of the model's emotional journey.

My approach is simple: start with Luna High and escalate when the task earns it. If the architecture is genuinely ambiguous, failures remain unexplained, or the work involves subtle security or concurrency behavior, I will reach for a stronger model. That is not model loyalty. It is routing.

The best model is not always the one that wins the benchmark. For agentic engineering, I care about the cost of the completed task, how long it took, how many retries it needed, whether the tests passed, and how much cleanup I had to do afterward.

For my workload, GPT-5.6 Luna with high reasoning effort currently hits the useful middle: very cheap, very fast, and accurate enough to handle the common path.

It is not the model I want solving every problem. It is the model I want holding the wrench most of the time.

Pricing sources: OpenAI model documentation and Anthropic pricing documentation, checked August 14, 2026.

I Asked Muse Glimmer to Pair Program With Me in OpenCode

Ahmed Nafies — Tue, 11 Aug 2026 22:04:13 +0000

I wanted to use Meta's Muse Glimmer as a local agentic engineering assistant inside OpenCode.

Not as a chatbot in a browser. I wanted an agent that could inspect a repository, understand a task, edit files, run tests, and explain what it did.

The command was refreshingly short:

ollama launch opencode --model muse-glimmer

The model, however, is a 30-billion-parameter coworker who arrives carrying an approximately 18 GB suitcase and expects a 128K-context office.

The onboarding meeting

First, I checked Ollama:

ollama --version

Then I started the local service if needed:

ollama serve

I verified that the local API was alive:

curl http://localhost:11434/api/version

Then I launched OpenCode with Muse Glimmer:

ollama launch opencode --model muse-glimmer

The first run downloads the model. This is a good opportunity to make coffee, reconsider your disk space, and remember that “local AI” does not mean “tiny AI.”

To inspect installed models:

ollama ls

On Apple Silicon, Ollama also lists an MLX variant:

ollama launch opencode --model muse-glimmer:30b-mlx

The standard model is listed at around 18 GB. That is the model's luggage, not the entire hotel bill. Leave memory for the context, OpenCode, the operating system, and the repository you are asking the agent to understand.

Why OpenCode changes the experiment

The interesting part is not just running Muse Glimmer locally. It is putting the model inside an engineering workflow.

With OpenCode, I can ask the agent to:

Inspect the repository.
Find the relevant files.
Explain the current behavior.
Propose a plan.
Make a focused change.
Run the project's tests.
Read failures.
Revise the patch.
Show the final diff and verification result.

That is the difference between asking a model for code and asking an agent to help complete an engineering task. The model can still be confidently wrong. The workflow gives me more chances to catch it before the wrongness becomes a pull request.

The first repository tour

I opened the repository in OpenCode and started with a read-only request:

You are working in a local software repository. Inspect the project structure and identify:

- The application entry point
- The test command
- The main source directories
- Configuration files that affect development
- Any obvious contribution or safety constraints

Do not modify files. Report the evidence and ask before making changes.

This is a better first task than “fix everything.” It tests whether Muse Glimmer can build context without immediately turning the repository into modern art.

The key signal is whether the response points to actual files and commands. A useful agent should report evidence, not invent a project architecture because the folder name sounded familiar.

The first engineering task

After the repository tour, I gave OpenCode a small, testable issue:

Find the bug in the relevant average function. Explain the edge case, propose the smallest fix, and add a regression test. Do not change unrelated files.

The current implementation is: average(values) = sum(values) / len(values) + 1

The extra + 1 is suspicious. It is the kind of bug that enters production wearing sunglasses and saying, “Nobody will notice.”

A good OpenCode session should locate the real implementation, find the project's existing test style, make the smallest patch, and run the relevant test. A locally generated patch is still a patch. It needs review, tests, and version control like any other change.

Planning before editing

I asked OpenCode to separate investigation from implementation:

Before editing, write a short implementation plan with:

- Files you expect to inspect
- The behavior you expect to change
- The test you will add or update
- The command you will run to verify the change

Wait for approval before modifying files.

This creates a useful checkpoint. It also prevents the classic coding-agent experience where you ask for one small fix and receive a surprise framework migration, a new abstraction layer, and a README poem.

OpenCode is the interface where this loop becomes practical: inspect the plan, approve the direction, watch the diff, and ask for verification in the same engineering session.

The test-and-repair loop

The useful workflow looks like this:

Inspect -> plan -> approve -> edit -> test -> inspect failure -> revise -> test again

I tested failure recovery with an intentionally broken test result:

The test runner returned:

ERROR: assertion failed in tests/test_average.py
Expected: 2.0
Received: 3.0

Explain the failure, identify the likely cause, and propose the smallest correction. Do not edit files yet.

Then I tested a repository constraint:

The repository check returned:

ERROR: the generated lockfile differs from the committed lockfile.

Do not regenerate dependencies. Explain what should be reviewed first.

The goal is not to make the agent retry everything until the terminal turns green. The goal is to see whether it can distinguish a fixable implementation error from a situation requiring human review.

“The model usually behaves” is not an engineering control. It is something you say immediately before opening an incident ticket.

Screenshot-driven debugging

Muse Glimmer supports image input, which is useful for UI bugs, screenshots, diagrams, and terminal output. From the local OpenCode workflow, I can give the agent a screenshot and ask for evidence-first analysis:

Inspect this screenshot as a debugging artifact. List only visible symptoms, separate observations from hypotheses, and suggest which source files or browser checks should be inspected next.

“The button is clipped” is an observation. “The CSS grid is broken because of a missing min-width” is a hypothesis. That distinction saves time and prevents a screenshot from becoming a license to rewrite the frontend.

Conclusion

Using Muse Glimmer with OpenCode through Ollama is straightforward to start:

ollama launch opencode --model muse-glimmer

The interesting work begins after OpenCode starts. Ask the agent to inspect, plan, edit, test, diagnose, and explain.

Muse Glimmer can be the clever local coworker. Ollama handles the model runtime. OpenCode handles the engineering conversation. You remain the person who decides whether the patch deserves to live.

Resources:

I Let an Autonomous Agent Build My RAG Stack. Here's How the Loop Worked

Ahmed Nafies — Thu, 06 Aug 2026 17:24:49 +0000

I Let an Autonomous Agent Build My RAG Stack. Here's How the Loop Worked

I didn't write a single line of code. I didn't debug a Helm chart. I didn't stare at a traceback wondering why the connector wouldn't authenticate.

I typed one sentence and watched.

Twenty minutes later, a draft pull request landed in my private repo. Everything behind that PR — the OpenSearch cluster, the embedding models, the five init jobs in lockstep, the MCP server, the RAG pipeline, and the 30 seeded articles — was produced by an autonomous coding agent using a workflow called the loop.

The stack itself is impressive enough: OpenSearch 3.7 with hybrid search fusing semantic and full-text results, a retrieval-augmented generation pipeline powered by a local Ollama LLM, a FastMCP server exposing four tools over SSE, and a Skaffold + Helm deployment that goes from zero to running cluster with skaffold run. But the stack isn't the story.

The story is what happened between the prompt and the PR. The planning, the self-correction, the moment the agent deleted 14 model chunks it thought were duplicates, broke the entire embedding system, and then — without any help — fixed it.

This is how the loop worked.

The loop isn't just an agent. It's a contract.

The loop is a skill for opencode that turns a natural-language prompt into a working feature branch and a draft PR. It sounds like every other AI coding tool until you see what it won't do: it won't write code before you approve a plan.

Phase	What happens
proposal	Explores the codebase, analyzes the prompt, proposes a plan. No implementation.
agreement	You review. You can push back, ask questions, or say go. The agent waits.
implementation	Builds the agreed scope on a feature branch, committing early and often.
draft_gate	Self-review: lint, tests, diff inspection. Fixes issues before you ever see them.
draft_pr	Pushes the branch, opens a draft PR, reports back. Done.

Every agent should work like this. Most don't. Most just start writing code the moment you press enter, and you spend the rest of the session saying "no, not like that."

The loop carries its state in .loop-state.json — the current phase, the branch, the PR number, the plan, and a log of every phase transition. If your session crashes or you walk away for an hour, the loop resumes exactly where it left off. This alone makes it viable for multi-hour autonomous runs.

Then there's loop-police: a companion skill that rides shotgun, watching for infinite loops. Thinking loops. Tool-call loops. File-read spirals. Stagnation. When loop-police detects the agent going in circles, it interrupts: "You're stuck. Pivot." That intervention is the difference between an agent that wastes ten turns retrying the same failing deploy and one that recognizes the failure and changes strategy.

Without loop-police, the chunk-deletion disaster I'll get to later might have been unrecoverable.

The plan was one paragraph

The agent's proposal:

Build opensearch-hybrid-mcp mirroring the os-hybrid-search reference: OpenSearch 3.7 with Dashboards, MCP server (FastMCP + UV), five init jobs (template → embedding → connector → pipeline → seed), skaffold.yaml with six images, justfile, README. Private repo + draft PR.

And then a diagram — the agent drew the architecture:

macOS host                             Kubernetes (Colima)
┌───────────────┐                      ┌────────────────────────────────────────────────────────────┐
│ Ollama        │                      │  ┌──────────────────┐  ┌────────────────┐                  │
│ (Desktop)     │◄──────── :11434 ────►│  │  OpenSearch 3.7  │  │  OS Dashboards │                  │
│ qwen3.5:9b-mlx│  host.docker.internal│  │  os-hybrid:9200  │  │  :5601         │                  │
└───────────────┘                      │  └────────┬─────────┘  └────────────────┘                  │
                                       │          │                                                 │
                                       │  ┌────────▼──────────┐                                     │
                                       │  │  MCP Server       │    FastMCP + UV                     │
                                       │  │  mcp-server:8000  │    hybrid_search                    │
                                       │  │                   │    rag_query                        │
                                       │  │                   │    index_document                   │
                                       │  │                   │    cluster_health                   │
                                       │  └───────────────────┘                                     │
                                       │                                                            │
                                       │  Init Jobs (run in sequence):                              │
                                       │    1. template-job    index template                       │
                                       │    2. embedding-job   all-MiniLM-L12-v2                    │
                                       │    3. connector-job   Ollama → Qwen 3.5                    │
                                       │    4. pipeline-job    chunking/hybrid/RAG                  │
                                       │    5. seed-job        30 articles                          │
                                       └────────────────────────────────────────────────────────────┘

Ollama runs on the host, not inside the cluster — pods reach it through host.docker.internal:11434. The agent figured that out by itself. Single-node OpenSearch, 10Gi PVC, no security (it's a dev cluster), MCP server on SSE port 8000. Five init jobs that run in sequence, each waiting for its upstream dependencies through retry loops.

I said yes. The agent started building.

The implementation was fast. The recovery was where it got interesting.

Seven commits. Forty-plus files. The agent scaffolded the entire project in minutes.

Six Helm charts (OpenSearch parent, MCP server, five init jobs). A skaffold.yaml with six Docker images building in parallel. A justfile with shortcuts for skaffold run, kubectl port-forward, and job logs. Every init job is a shell script with until loops — each waits for OpenSearch, for the index template, for the embedding model to deploy, for the pipeline to be ready. If a dependency isn't there, the job pauses. If it is, the job runs. This makes the whole stack re-runnable: delete a job and re-create it, and it picks up where it left off.

There's a satisfying recursion here. The loop is a five-phase workflow of gates and retries. The jobs it wrote are retry loops with dependency checks. The agent built code that mirrors its own architecture.

The MCP server landed next: FastMCP, four tools:

hybrid_search(query, index?, k?) — neural + full-text, z-score fused
rag_query(question, index?, k?) — retrieves context, generates answer
index_document(id, title, body, index?) — chunks + embeds + indexes
cluster_health() — cluster status

The agent ran its own draft gate: helm lint and helm template on seven charts, skaffold render, sh -n on every script. Clean. It pushed the branch, opened draft PR #1, and reported back.

Technically, the loop was done: proposal → agreement → implementation → draft_gate → draft_pr. Five phases, zero handoffs.

But the stack hadn't been deployed yet. And that's where everything went sideways — then sideways again — then somehow straightened out.

Five bugs. One autonomous agent. Zero panicking.

The first skaffold run kicked off and immediately hit a wall. The agent didn't wait for me to notice. It read the logs, diagnosed the problem, patched it, and redeployed. Five times.

Bug 1 — "Connector credential is null or empty list": OpenSearch 3.7 demands a credential object even for local Ollama endpoints that don't use authentication. The agent added a dummy key (openAI_key: "ollama-local") and — crucially — switched the connector from Ollama's native /api/chat to the OpenAI-compatible /v1/chat/completions, which is what the RAG processor expects. Two fixes in one pass. Redeployed. Connector created.

Bug 2 — The one-character regex disaster: RAG queries hit a 400: "Connector URL is not matching the trusted connector private endpoint regex". The agent had written ^http://host.docker.internal:11434:.*$ — a colon after the port. The actual URL was http://host.docker.internal:11434/v1/chat/completions — a slash. The difference between : and / broke the entire pipeline. The agent found it, flipped the character, and redeployed. The kind of bug a human would spend 20 minutes on. The agent caught it in one.

Bug 3 — "Model 'ollama-qwen3.5' not found": The connector was passing the OpenSearch-registered model name to Ollama, but Ollama expects the raw model name (qwen3.5:9b-mlx). The agent traced the call chain — OpenSearch RAG processor → connector → Ollama API — and updated the MCP server to pass the correct model name in ext.generative_qa_parameters. Fixed.

Bug 4 — Memory circuit breaker: Seed-job bulk indexing hit 429 rate limits. JVM heap at 93% on 2g allocation. The agent bumped it to 4g (-Xmx4g -Xms4g, container limits 4g/8Gi), redeployed, and the jobs succeeded. The agent understood enough about JVM memory pressure to recognize the root cause without being told.

Bug 5 — The chunk-deletion disaster: This is the one where the agent almost nuked the project. While cleaning up orphaned model registrations, it deleted 14 documents from the ML system index — thinking they were duplicates. They weren't. They were the embedding model's chunks (_0 through _13). The model went to DEPLOY_FAILED. The embedding pipeline was dead.

Most agents would either not notice, or notice and keep retrying the same broken state until you killed the session. The loop did something else: it recognized the mistake. It saw DEPLOY_FAILED on the model, understood that the chunks it deleted were critical, deleted the broken base model document, let the embedding-job re-register fresh with a new model ID, and re-ran the pipeline and seed jobs. The stack healed itself.

That pattern — deploy, hit error, read logs, diagnose, patch, redeploy, verify — repeated five times. The agent operated the stack as well as it built it.

The moment it worked

After the last fix, the cluster settled. Thirty articles seeded. Both models deployed. Cluster health green.

A hybrid search returned ranked results with fused neural and full-text scores. A RAG query against the pipeline — retrieves context first, then calls Ollama through the connector, returns the answer grounded in the retrieved documents:

{
  "query": { "match": { "body": "inverted index TF-IDF BM25" } },
  "ext": {
    "generative_qa_parameters": {
      "llm_model": "qwen3.5:9b-mlx",
      "llm_question": "Based on the context, explain what an inverted index is and how TF-IDF or BM25 ranking works."
    }
  }
}

The system answered:

An inverted index is a structure that maps each unique word to the documents containing it, allowing for fast retrieval based on query terms. Regarding ranking, TF-IDF measures term frequency versus inverse document frequency, while BM25 improves upon this method by saturating term frequency.

Grounded. Accurate. Sourced from the 30 articles the agent itself had seeded. All four MCP tools exercised and verified over SSE. The agent updated the draft PR with the validation results.

What I'd do differently (and what I won't)

Loop-police is not optional. Without it, the agent would have retried that failing deploy until the heat death of the universe. With it, the agent flagged the DEPLOY_FAILED state transition, recognized it had caused the failure, and pivoted to recovery. If you're running autonomous agents, you need a stall detector. Period.

The agreement gate is the whole point. Autonomous coding agents without a plan-agreement phase are just hyperactive interns with commit access. The loop doesn't move from proposal to implementation until you say yes. That one rule eliminates the most common failure mode of AI coding: the agent building the wrong thing while you watch helplessly.

State files make long-running sessions possible. .loop-state.json tracks phase, branch, PR number, and plan history. If the session drops — and with tools running 40+ minute autonomous sequences, it will — the loop resumes without context loss. This is infrastructure, not a nice-to-have.

Retry-loop init jobs are self-healing infrastructure. Each job waits for its dependencies through until loops. You can tear down a job and recreate it — it'll wait, detect its prerequisites are met, and proceed. The loop wrote infrastructure that works the way the loop itself works. That's either poetic or recursive. Probably both.

The draft gate is the QA step you'll never do yourself. The loop ran helm lint, helm template, skaffold render, and sh -n before opening a PR. No human on my team does that for every commit. The agent does it because the workflow demands it. Fail the gate, and the loop doesn't proceed to PR. This catches broken manifests before they reach the repo — before you even see them.

The loop isn't magic. It's process.

That's the real takeaway. Autonomous coding agents don't fail because they're not smart enough. They fail because they lack structure — no plan phase, no review gate, no self-check, no stall detection.

The loop adds that structure. It turns an agent from a code generator into a collaborator: you get a plan, you approve it, the agent builds, it self-corrects, and it ships a draft PR. You review, you merge, you move on.

I didn't write a single line of code. But the loop didn't build my RAG stack by accident. It built it because the process forced it to plan, verify, and recover — the same way a good engineer would.

That's how the loop worked.

Building a 100% Local RAG System on Kubernetes — No API Keys Required

Ahmed Nafies — Thu, 30 Jul 2026 23:00:55 +0000

Here's what we're building:

The itch I couldn't scratch

Last week I sat down to build a RAG system. You know, one of those "chat with your documents" things everyone's building these days.

I opened my first tutorial. It said: "Step 1: Sign up for OpenAI and grab your API key." Fine. "Step 2: Create a Pinecone account." Okay. "Step 3: Deploy to a managed vector database." Hmm. "Step 4: Set up your billing — here's a pricing calculator."

I stared at the calculator. Then at my $0 budget. Then back at the calculator.

Something felt off. Why does a "personal project" need a procurement department? Why does my data need to leave my laptop to answer a question about a document sitting on my laptop? And honestly — I just wanted to tinker. Not negotiate with three SaaS vendors before I could write a single line of code.

So I did what any reasonable engineer would do. I closed all twenty tabs, opened a terminal, and decided to build the whole thing from scratch. On my machine. With models running on my CPU. No cloud. No API keys. No credit card.

Here's how that ride went.

The game plan

Before writing a line of code, I sketched out what "no cloud" actually means:

Embeddings — gotta run locally. No calling OpenAI's embedding endpoint.
Vector search — needs to live inside something I already run. I'm not spinning up a separate vector database service.
The LLM — must fit in my laptop's RAM and run on CPU. I don't have a GPU lying around.
Deployment — real infrastructure patterns, not docker-compose up and pray. Something that would make sense in production, just scaled down.

What emerged was this stack:

┌────────────────────────────────────────────────────┐
│                 FastAPI Application                 │
├────────────────────────────────────────────────────┤
│  ┌──────────────┐  ┌──────────┐  ┌──────────────┐  │
│  │  Embedding   │  │ pgvector │  │  llama-cpp   │  │
│  │  Pipeline    │  │  Search  │  │  (Qwen 1.5B) │  │
│  └──────────────┘  └──────────┘  └──────────────┘  │
└────────────────────────────────────────────────────┘
                         │
                         ▼
               ┌──────────────────┐
               │   PostgreSQL     │
               │   + pgvector     │
               │   + HNSW index   │
               └──────────────────┘

The flow: someone asks a question → it gets turned into a 384-dimensional vector → PostgreSQL hunts for similar documents using cosine similarity → those results get stuffed into a prompt → the LLM answers → answer streams back. All inside Kubernetes. All local.

Every piece of this puzzle was chosen with one rule: it runs on my machine, or it doesn't make the cut.

Choosing the players

pgvector: the database you already have

I wanted vector search without installing Yet Another Database. pgvector is a PostgreSQL extension that adds vector columns, indexing, and similarity operators directly into Postgres. Just CREATE EXTENSION IF NOT EXISTS vector and you're in business.

It gives you HNSW indexes (fast approximate nearest neighbor search), a <=> cosine distance operator, and zero additional infrastructure. Your vector DB is just... your regular DB.

Qwen 2.5, living in my CPU

Here's where things got spicy. Most RAG tutorials assume you'll call GPT-4. But I had no GPU, no cloud credits, and a strong desire to keep everything local.

Enter Qwen2.5-1.5B-Instruct — a 1.5 billion parameter model that actually runs on CPU. Not fast, but fast enough. And the output quality? Genuinely surprising for its size.

The trick is serving it via llama-cpp-python, which exposes an OpenAI-compatible API at http://127.0.0.1:8001/v1. The magic of this: my application code uses the exact same OpenAIChatModel class you'd point at api.openai.com. It doesn't know — and doesn't care — that the model is running on the same machine, on a CPU, inside a container.

All it sees is a /v1/chat/completions endpoint. The fact that it's a 1.5B model crammed into a Docker container? That's between me and the container.

sentence-transformers: tiny but mighty

For embeddings, I picked all-MiniLM-L6-v2. It produces 384-dimensional vectors, runs fast on CPU, and the quality is solid for semantic search. It gets loaded once at startup and cached — no reloading per request:

from sentence_transformers import SentenceTransformer

@lru_cache
def get_model() -> SentenceTransformer:
    return SentenceTransformer("all-MiniLM-L6-v2")

def embed_text(text: str) -> list[float]:
    return get_model().encode(text).tolist()

Kubernetes on a laptop? Absolutely.

Running K8s locally sounds excessive until you try it. I used Colima — a lightweight k3s cluster that runs on 2 CPUs and 16GB of RAM. No Docker Desktop, no resource vampires. Just a clean K8s cluster on my Mac.

The benefit is real: Helm charts for declarative infra, Skaffold for the build-deploy loop, Helm hook Jobs that run migrations and seeding automatically. The same patterns I'd use in production, just at my desk.

Under the hood

Let's walk through the interesting bits. Not every line — just the parts that made me smile when they worked.

The database that thinks in vectors

The documents table stores each document alongside its 384-dimensional embedding vector. The HNSW index means similarity searches stay fast even as the corpus grows:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE documents (
    id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),
    title       varchar(500) NOT NULL,
    content     text NOT NULL,
    embedding   vector(384) NOT NULL,
    created_at  timestamptz DEFAULT now()
);

CREATE INDEX ON documents
    USING hnsw (embedding vector_cosine_ops)
    WITH (m = 16, ef_construction = 64);

The search that feels like magic

A stored procedure does the heavy lifting. You throw a vector at it, and it returns the closest matches ranked by similarity:

CREATE OR REPLACE FUNCTION match_documents (
    query_embedding vector(384),
    match_threshold float,
    match_count int
)
RETURNS TABLE (id uuid, title text, content text, similarity float)
LANGUAGE sql STABLE
AS $$
    SELECT
        documents.id,
        documents.title,
        documents.content,
        1 - (documents.embedding <=> query_embedding) AS similarity
    FROM documents
    WHERE documents.embedding <=> query_embedding < 1 - match_threshold
    ORDER BY documents.embedding <=> query_embedding
    LIMIT match_count;
$$;

That <=> is pgvector's cosine distance operator. 1 - distance gives us similarity. Clean, fast, entirely in SQL.

The application calls it through a parameterized SQLAlchemy query — no ORM magic, just raw SQL with safe parameters:

async def match_documents(
    session: AsyncSession,
    query_embedding: list[float],
    match_threshold: float = 0.5,
    match_count: int = 5,
) -> list[dict]:
    query = text("""
        SELECT * FROM match_documents(
            CAST(:query_embedding AS vector(384)),
            :match_threshold,
            :match_count
        )
    """)
    result = await session.execute(query, {
        "query_embedding": embedding_str,
        "match_threshold": match_threshold,
        "match_count": match_count,
    })
    return [{"id": str(r["id"]), "title": r["title"],
             "content": r["content"], "similarity": r["similarity"]}
            for r in result.mappings().all()]

The agent that thinks it's talking to OpenAI

This is my favorite part. Pydantic AI's Agent class is designed for OpenAI, but llama-cpp-python speaks the same protocol. So we just... point it at localhost:

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    "qwen2.5-1.5b-instruct",
    provider=OpenAIProvider(
        base_url="http://127.0.0.1:8001/v1",
        api_key="not-needed",
    ),
)

agent = Agent(model=model, system_prompt=(
    "You are a helpful assistant. Answer questions based on "
    "the provided context. If the context does not contain "
    "enough information, say so."
))

The api_key="not-needed" line makes me happy every time I look at it. Take that, billing page.

Streaming: the one that nearly broke me

Server-Sent Events streaming should be straightforward. You yield chunks, the browser displays them. Simple.

Except Pydantic AI's stream_text() doesn't return deltas — it returns the accumulated text. Every chunk is the full response so far. If you send that to the browser, it keeps re-rendering the entire message.

The fix is deceptively simple. Track the previous value and compute the difference:

async def generate():
    yield f"event: sources\ndata: {json.dumps(sources)}\n\n"

    prev = ""
    async with agent.run_stream(prompt) as result:
        async for chunk in result.stream_text():
            delta = chunk[len(prev):]  # Just the new stuff
            prev = chunk
            yield f"event: delta\ndata: {json.dumps({'content': delta})}\n\n"

return StreamingResponse(generate(), media_type="text/event-stream")

Two lines of code that took two hours to figure out. But watching the text flow in, word by word, on the first successful stream — chef's kiss.

Ship it: Kubernetes all the way down

Three Helm charts, one skaffold run:

Chart	What's inside
`rag-supabase-db`	PostgreSQL StatefulSet + pgvector, 5Gi persistent disk
`rag-supabase`	FastAPI + llama-cpp running side-by-side, 3Gi model cache
`rag-supabase-ui`	Nginx serving the chat UI

The slick part: migrations and seeding aren't manual steps. They're Helm hook Jobs that fire automatically on every deploy:

apiVersion: batch/v1
kind: Job
metadata:
  name: {{ .Release.Name }}-migrate-{{ .Release.Revision }}
  annotations:
    "helm.sh/hook": post-install,post-upgrade
    "helm.sh/hook-weight": "0"
    "helm.sh/hook-delete-policy": before-hook-creation
spec:
  template:
    spec:
      restartPolicy: OnFailure
      containers:
        - name: migrate
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          command: ["uv", "run", "alembic", "upgrade", "head"]

The migrate Job fires first (hook-weight: "0"), then the seed Job (hook-weight: "10"). The seed Job is smart — it checks if the documents table already has data and skips gracefully. No more "did I already run the seeder?" anxiety.

And the LLM model? It downloads a ~1GB GGUF file on first startup and caches it in a PVC at /models. First deploy takes a few minutes while it downloads. Every deploy after that? Instant. The model survives pod restarts, rebuilds, everything.

War stories from the trenches

No project worth building comes without battle scars. Here are the ones that left marks.

The ghost in the database

After deploying, everything worked perfectly. Come back the next morning, and the app is dead. PostgreSQL connections had silently died during idle periods.

The fix: pool_pre_ping=True and pool_recycle=300 on the SQLAlchemy engine. It pings connections before using them and recycles them every 5 minutes. Simple fix, but diagnosing it involved a lot of confused staring at error logs.

The streaming trap

I already mentioned this, but it deserves its own section. I spent two hours debugging why my streams were "working" but the browser kept re-rendering the entire message every chunk. The culprit: stream_text() returns cumulative text, not incremental deltas. The chunk[len(prev):] trick is now burned into my brain forever.

Dependency hell, one version at a time

Pydantic AI 0.8.1 depends on opentelemetry._events — a module that OpenTelemetry removed in version 1.44. The fix? Pin opentelemetry-api>=1.42,<1.43 and move on with your life. Version pinning: not glamorous, but it keeps the lights on.

Skaffold, Helm hooks, and the great image tag mystery

Skaffold is amazing at building images and deploying Helm charts. But Helm hook Jobs? They reference images through .Values.image.tag, and Skaffold doesn't inject its build tags into hook templates.

The workaround: always tag your image as latest and match it in values.yaml. Not elegant, but it works. I'll fix it properly someday. (I won't.)

The first deploy is always the slowest

That first skaffold run takes longer because the container is downloading a ~1GB GGUF model. But that's the beauty of Persistent Volumes — the model sticks around. Every subsequent deploy is fast, because the model is already sitting in /models, warm and waiting.

So... does it work?

Yes. Beautifully.

Type a question. Watch the embedding get computed locally. See PostgreSQL find the relevant documents in milliseconds. Read the LLM's response as it streams in, character by character. Check the source citations to see which documents informed the answer.

And here's the thing that still gets me: all of it is happening on my laptop. No packets leaving my network. No tokens being counted by a billing system somewhere. No vendor lock-in. Just my code, my models, my data.

The pipeline handles everything end-to-end:

Natural language questions → local embedding
Semantic search → PostgreSQL with pgvector and HNSW
Retrieved context + question → local LLM via llama-cpp-python
Streaming response → back to the browser via SSE

One skaffold run deploys the entire stack. One kubectl port-forward lets you chat with it.

What I learned (and what's next)

This project changed how I think about AI infrastructure. We've gotten so used to reaching for managed services that we forget how much is possible on a single machine. A 1.5B parameter model isn't going to write your novel, but for RAG? For answering questions grounded in your own documents? It's more than enough.

The whole stack — PostgreSQL, pgvector, FastAPI, sentence-transformers, llama-cpp-python, Kubernetes — fits comfortably in 16GB of RAM. No GPU. No cloud. No API keys taped to the bottom of your keyboard.

Could it be better? Always. A better model, hybrid search (semantic + keyword), persistent conversation storage, zero-downtime deployments. That's for another weekend.

For now, I'm just enjoying the quiet satisfaction of asking my local LLM a question and watching it answer — powered by nothing but my CPU and a few hundred lines of code.

Repository: github.com/nf1s/rag-supabase

If you build something similar, tag me on Dev.to. I'd love to see what you cook up without a credit card. And if you run into the same streaming bug — chunk[len(prev):]. You're welcome.

DEV Community: Ahmed Nafies

I built an agentic coding exam for Hy4 preview, then made the model sit it

The test bed

The run

Two bugs the fixed tests did not catch

1. X | None is not Optional[X]

2. The one the fuzzer found

The disagreement

What I learned

Reusing this

So, does it impress me?

When I'd use it, and when I wouldn't

I'm using GPT Sol and Claude Opus for free — pi-coding-agent + OmniRoute

What "free" means here

1. Install both

2. Add OmniRoute as a Pi provider

3. Make it the default

4. Run it

Two gotchas that cost me time

Why this beats juggling provider keys

Testing OmniRoute locally: verify your AI gateway actually works before you trust it

1. Install and boot it

2. Run local diagnostics (omniroute doctor)

3. Verify the model list

4. Smoke-test a real request

5. Test streaming

6. Test tool calling

7. Test fallback (the whole point)

8. Point a real tool at it

Verification checklist

When a check fails

Testing Qwen3.8 Max on a Budget with OpenCode Go: 10 Tasks, ~10¢, Zero Failures

The budget math

Setup

The test

Results

Findings

1. It's fast because it doesn't "think"

2. The cache-write surprise (the big one)

3. The quality bar is high

4. Cost is dominated by caching, not difficulty

The cost breakdown

Verdict

Testing GLM-5.3 on a Budget with OpenCode Go: 10 Tasks, 4¢, Zero Failures

The budget math

Setup

The test

Results

Findings

1. It's a reasoning model — and it thinks a lot

2. Latency is the real price, not dollars

3. There's a hidden reasoning dial

4. The quality bar is genuinely high

The cost breakdown

Verdict

Why GPT-5.6 Luna High Is My Default for Agentic Engineering

The cheap model that gets the job done

Why I use high reasoning effort

And what about Claude Sonnet?

I Asked Muse Glimmer to Pair Program With Me in OpenCode

The onboarding meeting

Why OpenCode changes the experiment

The first repository tour

The first engineering task

Planning before editing

The test-and-repair loop

Screenshot-driven debugging

Conclusion

I Let an Autonomous Agent Build My RAG Stack. Here's How the Loop Worked

I Let an Autonomous Agent Build My RAG Stack. Here's How the Loop Worked

The loop isn't just an agent. It's a contract.

The plan was one paragraph

The implementation was fast. The recovery was where it got interesting.

Five bugs. One autonomous agent. Zero panicking.

The moment it worked

What I'd do differently (and what I won't)

The loop isn't magic. It's process.

Building a 100% Local RAG System on Kubernetes — No API Keys Required

The itch I couldn't scratch

The game plan

1. `X | None` is not `Optional[X]`

2. Run local diagnostics (`omniroute doctor`)