DEV Community: Azan Hyder

PromptLoop

Azan Hyder — Tue, 28 Jul 2026 13:44:37 +0000

azank1 / loop-llm

MCP server that observes every prompt, scores quality in real time, and closes the loop with iterative refinement. Built on FastMCP, SQLite, and Bayesian priors — no extra LLM required.

PromptLoop

Your agent said it works. PromptLoop is the receipt.

Cursor, Copilot, Claude Code — every agent ends a task with "done, all tests pass." Sometimes that's true. PromptLoop (package: loopllm) sits between your IDE agent and the model, scores each step the agent submits through two independent channels (a deterministic floor plus an LLM critic — the stricter score wins) and stamps the verdicts onto your git history as an audit trail you can review export, or gate CI on. No API key, no cloud, nothing leaves your machine.

What a caught lie looks like (real tool output):

loopllm_loop_start(goal="make the failing test pass", task_type="bugfix")
  → { suggested_budget: 3, quality_threshold: 0.8, evaluator_type: "composite" }
loopllm_loop_step(session_id, step_output="pytest: 3 failed, 12 passed")
  → {
      decision: "continue",
      score: 0.0,
      channel_a_score: 0.0,      ← deterministic floor: "tests passed" not found
      channel_b_score: 0.55,     ← LLM critic was more forgiving — vetoed
      deficiencies: ["Required pattern not

…

View on GitHub

Every coding agent ends a task the same way: "done, all tests pass."

Sometimes that's true. Sometimes the tests didn't pass. Sometimes the agent re-ran nothing, changed two lines, and declared victory because the loop hit its iteration budget — or worse, because it graded its own task.

Today I ran PromptLoop against my own agent in Cursor, on my own repo, and let it veto the agent's claims in real time. This is what happened, with the actual scores.

The moment it mattered

I gave the agent a bugfix goal and let it run. On step 1, the agent's posture was "tests look fine enough to ship." The artifact it submitted told a different story: 2 failed, 3 passed.

Here's the verdict that came back:

step 1  score=0.333  CONTINUE  Required pattern not found: tests passed

The agent reported progress. The receipt said no. The loop kept going.

Step 2 came back greener but still incomplete — 0.667, CONTINUE. Step 3 finally submitted a real, full pytest log with the pass pattern present:

step 3  score=1.000  STOP  Goal reached ≥ 0.80

Trajectory: 0.333 → 0.667 → 1.000. The loop didn't stop when the agent felt done. It stopped when the evidence cleared the bar.

How the veto works

PromptLoop is an open-source MCP server that sits between your IDE agent and the model. Agents don't get to say "done" — they submit artifacts (test logs, diffs, summaries), and each one is scored through two channels:

Channel A — a deterministic floor: regex patterns, JSON validity, completeness checks. It can't be argued with.
Channel B — an independent LLM critic.

The final score is min(A, B) — either channel can veto. Optimism doesn't get a vote.

Full honesty on this capture: in today's session, MCP sampling for Channel B wasn't available, so this run was cdv_mode: channel_a_only. The deterministic floor alone still caught the false "done." I'm telling you this because the raw JSON says it — and because a tool whose entire pitch is receipts doesn't get to overclaim.

It also fails your bad prompts at the door

The other half of the product is Prompt Lab — a live prompt-quality scratchpad in the sidebar. Think ESLint, but for the ask itself. From today's session, same machine, real scores:

Prompt	Grade	Score
`write me some code`	F	38%
Add exponential backoff retry…	D	51%
Full spec: backoff + raise + tests + type hints + pytest	B	72%

38% → 72% by adding constraints, not vibes. Bad prompts fail in the lab instead of in production, where they cost you tokens and a debugging session.

The audit trail

Every verified step is stamped with the git commit that was HEAD at the time. loopllm audit --since origin/main gives you a reviewable report of what the agent did on your branch and how each step was verified. loopllm audit-gate reads it back in CI — no verified receipt, no merge.

Under the hood: Bayesian priors per repo learn how many steps your bugfixes actually take, stored in per-project SQLite. Local-first, no API key, nothing leaves your machine.

What broke while dogfooding

Because receipts cut both ways:

The MCP server launched with cwd=$HOME and started writing receipts under the wrong project ID — Loop Monitor looked empty until I fixed project scoping to prefer WORKSPACE_FOLDER_PATHS. If you upgrade, restart MCP.
Channel B needs host sampling support; on a host without it you get Channel A only. Still useful — the floor still vetoes — but it's not full dual-verify, and I won't screenshot it as if it were.

Both are documented in the README under Known Limitations.

Try it

pipx install "loopllm[mcp]"
loopllm install-mcp --ide cursor --rules

VS Code Marketplace: loopllm Prompt Gauge. Works with Cursor, VS Code, and Claude Code. MIT licensed.

azank1 / loop-llm

MCP server that observes every prompt, scores quality in real time, and closes the loop with iterative refinement. Built on FastMCP, SQLite, and Bayesian priors — no extra LLM required.

PromptLoop

Your agent said it works. PromptLoop is the receipt.

What a caught lie looks like (real tool output):

loopllm_loop_start(goal="make the failing test pass", task_type="bugfix")
  → { suggested_budget: 3, quality_threshold: 0.8, evaluator_type: "composite" }
loopllm_loop_step(session_id, step_output="pytest: 3 failed, 12 passed")
  → {
      decision: "continue",
      score: 0.0,
      channel_a_score: 0.0,      ← deterministic floor: "tests passed" not found
      channel_b_score: 0.55,     ← LLM critic was more forgiving — vetoed
      deficiencies: ["Required pattern not

…

View on GitHub

If your agent has ever gaslit you about a passing test, a ⭐ tells me this problem is as common as I think it is. And if you dogfood it and catch a false "done" of your own — open an issue with the receipt. Those are my favorite kind.

SDCA-Raqqr

Azan Hyder — Tue, 28 Jul 2026 13:43:38 +0000

The model: RAQQR — Regressed Asymmetric Quantile... (fit on log₁₀ price vs ln time since genesis).
7 quantile bands from Q1% to Q99%. The tails get different curvature — because BTC's upside and downside tails don't behave the same.
Bands are a pure function of the date. Deterministic, reproducible, no black box.

azank1 / sdca-raqqr

Bitcoin valuation and DCA backtest library. Implements the RAQQR asymmetric tail-curvature rainbow model to score market risk (0–100) and simulate accumulation/distribution strategies against historical price data.

sdca-raqqr

Bitcoin RAQQR Rainbow Valuation + Risk-Scored DCA Backtest Library

Bitcoin valuation and DCA backtest library. Implements the RAQQR asymmetric tail-curvature rainbow model to score market risk (0–100) and simulate accumulation/distribution strategies against historical price data.

The numbers this library produces match the reference HTML artifact's default configuration — proven by a cross-implementation parity test, not assumed.

Interactive Dashboard

Live at → sdca-raqqr.streamlit.app

What it does

Valuation bands — computes 7 RAQQR quantile price bands (Q1%→Q99%) for any date from 2009 onward; these are the statistical "rainbow" rails
Risk score — maps today's price against the bands to a 0–100 EQM risk number (low = historically cheap, high = historically expensive)
Composite signal — blends the EQM z-score with optional extra indicators (Sharpe, CBPL) into a single composite_risk value
DCA backtest — simulates a piecewise-linear allocation curve: buy X% of cash when risk is low, sell Y% of BTC when…

View on GitHub

Today's reading: price is sitting below the Q1% band.
EQM risk score: 1.0 / 100 — the most extreme "historically cheap" print the model can produce.

gauge

The risk score maps price position between the bands onto 0–100 in log space:
🟢 <25 accumulate · 🟡 25–50 watch · 🟠 50–75 caution · 🔴 >75 distribute
One number, updated daily, live on the dashboard.

Then the fun part: an editable allocation curve.
"Buy X% of my cash when risk is 10. Sell Y% of my BTC when risk is 90."
Drag the curve, re-run the backtest, compare against lump-sum. Conservative / Default / Aggressive presets included.
bactest

the caveats
Coefficients are full-sample fits → backtests are in-sample, illustrative, NOT a track record • No fees/slippage modeled yet • The score saturates at the extremes. It's in the README. DYOR, always. Not financial advice.
the quant devs; parity-tested to floating point
The repo's least flashy and most important feature: the Python core is parity-tested against the original JavaScript implementation to rtol = 1e-12. A Node script dumps golden band values from the verbatim JS formula; pytest asserts the Python port matches to floating point. One coefficients file is the single source of truth for both implementations, so the web frontend and the Python library can never silently drift apart.
Ten tests cover the parity contract, the signal interpolation against an independent reference implementation, and the backtest's compounding math against closed-form results.
bash

// Detect dark theme var iframe = document.getElementById('tweet-2080234056458080490-700'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2080234056458080490&theme=dark" }
Check Out the article on X ^

The five properties are only my draft, which one would you cut?

Azan Hyder — Tue, 28 Jul 2026 13:37:49 +0000

Azan Hyder

Jul 28

Composer Harness

#ai #programming #productivity #python

4 min read

Composer Harness

Azan Hyder — Tue, 28 Jul 2026 13:20:19 +0000

Field notes on the missing layer of the agent stack.

The fastest standardization in dev tool history

Eighteen months ago, "how does an agent call a tool?" had no standard answer. Every framework invented its own. Then the industry speedran one:

November 2024. Anthropic open sources MCP.
March 2025. OpenAI adopts it across ChatGPT and the Agents SDK.
April 2025. Google announces A2A with 50+ launch partners.
August 2025. IBM's ACP merges into A2A. The protocols are already consolidating.
Late 2025. Both MCP and A2A land at the Linux Foundation.

By December 2025, Anthropic's ecosystem update was counting 10,000+ active public MCP servers and 97M+ monthly SDK downloads. The official registry hovers near 10K records. Around 16,000 GitHub repos carry the mcp-server topic. A2A passed 150 supporting organizations by April 2026.

The protocols won. Faster than REST won, by most measures.

And yet every real stack is held together by hand

Here is the thing the adoption numbers hide: nobody runs one protocol.

Industry survey data puts the typical MCP environment at 2 to 7 configured servers. A serious deployment also runs long lived agent to agent tasks, A2A's entire reason to exist, since MCP is the wrong shape for stateful, multi turn delegation. Then there is the long tail of software that has no API at all and gets driven through computer use.

Each of those pairs is connected by hand written glue. Bespoke routing. Bespoke auth handoffs. Bespoke output parsing. Bespoke retry logic. Every team writes it, every team maintains it, none of it is shared.

The community saw this coming. When A2A launched, the top reaction was essentially: "We already have MCP. Why do we need another standard?" The honest answer is that they solve different halves of the problem. The honest problem is that nobody standardized the layer that composes them.

And it is not a theoretical pain. Zuplo's State of MCP 2025 report found security and access control is the #1 blocker to MCP in production, ahead of any capability gap. The tools work. Operating them together doesn't.

The bottleneck moved, and almost nobody followed it

The most useful idea in the 2026 "loop engineering" discourse is this:

In any agent loop, the verifier is the bottleneck. Not the model.

Generation is cheap and spins forever. The loop produces value only as fast as something can judge ship it, retry, or stop. Andrew Ng talks about evaluators. Karpathy frames the whole craft as generation and verification. The practitioners writing about loop engineering all land on the same gate.

Now multiply that by composition.

One agent, one loop, one verifier. Hard, but understood. N agents across M protocols inside a single goal? The verification surface explodes. Which step failed? Was the output even structured? Did the right agent get called? Was it allowed to do that? Single agent loop tools don't answer these questions, because they were never asked to.

Chat output makes it worse

Even when a multi agent run works, the result arrives as prose.

A chat bubble can't be checked against a schema. It can't be diffed between runs. It can't be monitored. It evaporates the moment you scroll. If the verifier is the bottleneck, then output you can't verify is output you can't trust. Prose is the least verifiable format our industry has ever shipped at scale.

Structured output is not a UX preference. It is a precondition for verification.

So what would the missing layer look like?

Sit with the problem long enough and a spec starts writing itself. MCP and A2A standardize how a message gets from A to B. They say nothing about how a dozen independently running agents get planned into one goal, called in the right order, authenticated, verified, and rendered. That missing layer is an agent harness. Here is what it would need:

Protocol agnostic dispatch. MCP and A2A today, n8n or OpenAPI or LangGraph servers tomorrow, through thin bridges that inherit auth, retries, and payments instead of rebuilding them per protocol. MCP and A2A should be just the first two bridges.
Discovery treated as retrieval. Not "stuff every agent into the context window and pray." Search. Filter, rank, inject only the relevant agents as tools, only when this turn needs them.
Verification as pipeline policy, with receipts. Structural checks today, semantic judging tomorrow, always explicit about which one you are getting. Every step gets a verdict, and a finished run should be exportable as evidence, not just a transcript you have to trust.
Structured output contracts. Dashboards, not chat bubbles. Output that persists, and output you can check.
Identity and scope as contract. Agents that declare what they are allowed to do, in a versioned spec that can't break underneath the people building on it.

Nothing on this list is a model problem. It is orchestration, observability, and contract design.

The question

The protocols standardized in eighteen months. The composition layer is roughly where MCP was in late 2024: everyone needs it, nobody has named it, and the glue code piles up.

So I will ask you what I have been asking myself:

What would you put in the harness layer? What is the property I am missing that would make multi protocol agent runs trustworthy in production? And what is on my list that you think is wrong?

I have been building one answer for a while. More on that this week.

Sources and further reading:

Model Context Protocol, Wikipedia (launch timeline, OpenAI adoption, Linux Foundation move)
The State of MCP and A2A in 2025, O. Glukhov (10K+ servers, 97M monthly downloads, A2A ecosystem count, the "we already have MCP" debate)
MCP usage statistics roundup, MCPManager
State of MCP 2025, Zuplo (security as the #1 production blocker)
Loop Engineering in AI Agents, AI Builder Club, 2026 (the verifier is the bottleneck frame)