DEV Community: Michael Hairetis

I spent a week building an optimization, A/B tested it, and deleted it

Michael Hairetis — Mon, 28 Sep 2026 11:57:00 +0000

I had a 25,000-token system-prompt overhead on every agent call. Obvious problem, obvious solution: build a session-reuse layer so the overhead is paid once and amortised across many calls.

So I built it. Session pooling, lifecycle management, identity tracking, the works. It was the most intricate thing in that part of the codebase and I was quietly pleased with it.

Then I A/B tested it against not having it.

No difference.

The provider's prompt cache hits on prefix content regardless of session identity. Two completely independent calls inside the cache window already got the benefit. The optimisation I had carefully engineered was buying something I already had for free, and had been getting for free the entire time I was building the thing to get it.

I deleted the whole layer.

The lesson is not "measure before optimising." Everyone says that and everyone nods and nobody does it, because the optimisation is obviously going to help.

The sharper version is this: when your semantics are stateless, write stateless code. I had reached for a stateful design to solve a problem the platform had already solved statelessly. The complexity was not buying performance, it was buying a mental model that did not match the system underneath.

Three other things from the same project, all in the same shape:

Structure beats trust. A single agent given a complex task fails unpredictably. It loses subgoals and declares victory early. Rather than trying to make one agent reliable, the pipeline does the managing and catches failures at phase boundaries.

It is a routing problem. I had one heavyweight backend answering both "which of these three pipelines" and "go do two minutes of multi-step research." Making a heavyweight agent answer a classification question costs heavyweight money for a featherweight answer. Splitting them made both paths faster and cheaper.

Old heuristics still earn their keep. Asking a model "is this task finished" costs about five cents a check. Pattern matching on the agent's own activity signals, with the model as fallback for genuinely ambiguous cases, cut average per-request cost roughly tenfold.

None of these were model problems. None were solved by a better prompt.

The full write-up is on my site, including why I think the infrastructure half is the half that actually decides whether the thing works in production.

https://openred.space/blog/the-boring-parts-of-agentic-ai.html

98.7% better on seasonal, 9.8% worse on a random walk: the most useful benchmark I ran

Michael Hairetis — Sat, 26 Sep 2026 12:29:59 +0000

Before pointing Google's TimesFM 3.0 at anything real, I ran a calibration baseline. Two synthetic series with known properties, same model, same settings, held out.

Seasonal with drift: 98.7% better than the naive baseline, 100% direction accuracy.
Random walk: 9.8% worse than naive, 47% direction.

The naive baseline is the crudest forecast available: next value equals last value.

Row two is the important one, and it is not a defect.

A random walk is unpredictable by construction. Each step is independent of every step before it. A model that appeared to forecast one would be reporting structure that does not exist. In a backtest that looks like skill. In production it looks like losing money.

The correct behaviour on an unpredictable series is to fail, ideally about as badly as naive. That is what happened.

Read together the two rows are a map.

Where it works: demand forecasting, energy load, web traffic, call volume, inventory, capacity planning, sensor telemetry. Domains where next week resembles last week plus a trend.

The commercial argument there is not really accuracy, it is that you do not fit a model per series. Ten thousand SKUs, ten thousand zero-shot forecasts, about 8 ms each.

Where it does not: anything closer to a random walk than to a seasonal series.

I then spent two days confirming that market tape sits firmly on the second side of that line.

https://openred.hashnode.dev/a-forecasting-model-that-correctly-refuses-to-forecast?utm_source=hashnode&utm_medium=feed

Your menu needs an API, and the hard part is not the payment

Michael Hairetis — Fri, 25 Sep 2026 22:39:00 +0000

Direct ordering has existed for years and most independent restaurants still send most of their off-premise volume through marketplaces at 15% to 30% commission. There is a good reason, and agents change it.

The objection that mattered: direct ordering made you responsible for your own demand. The marketplace was a place customers already were, with a recommendation engine and a promotions budget. Going direct meant keeping 15 to 30 points and inheriting a marketing job you had no staff for. A high-margin channel nobody uses loses to a low-margin channel that is full.

When a customer tells an assistant to order from a named restaurant, the demand routing is already done. The agent is looking for an endpoint that can take the order. If you have one it goes there. If not it falls back to whoever does, and you pay commission on demand you had already earned.

Direct ordering stops being a marketing problem and becomes an integration problem. Integration problems are tractable.

The build list, honestly:

A catalogue as structured data. Items, modifiers, prices, tax treatment, real availability, prep time. The single most common blocker, because most independents do not have an accurate machine-readable menu anywhere, including inside their own POS.
A quote endpoint. Priced basket including tax and fees, with a short TTL. Not the same as publishing a menu, and where naive integrations break.
Payment acceptance. Two live standards on HTTP 402: x402 (Coinbase, stablecoin-native, most volume) and MPP (Stripe and Tempo, also routes cards via Shared Payment Tokens). You do not get to pick which one the customer's agent speaks, so sit behind a processor that handles both.
Order injection into the POS. Toast, Square and Clover all have APIs and all differ. This is where the hours go, and why "add an ordering page" is not the same project.
A status channel back to the agent.
Discoverability in whatever registry agents consult.

The part nobody has built: exceptions. Out of stock, substitution approval, partial refund on a payment that already settled. The payment protocols solve payment and say nothing about the commercial relationship around a failed order. Ask anyone selling you agent-readiness how they handle it.

https://openred.space/blog/your-menu-needs-an-api.html

I built the classifier tier in April. A model just shipped to be it.

Michael Hairetis — Thu, 24 Sep 2026 16:06:22 +0000

TypeSafe AI came out of stealth with Jev, a model that does not generate text. You hand it state and a typed question, it returns a calibrated probability over a fixed answer set. Nothing to parse, no invalid answer possible.

My reaction was not "clever idea." It was "I have had that in production since April."

That reaction is partly wrong and worth unpacking.

What I will not claim: using a model as a classifier is not novel and was not novel in April. It has been ordinary since function calling shipped, and calibrated-label training sits in a research lineage older than any of this. Arriving somewhere independently is not arriving first. And Jev is a different category from what I built: I implemented a pattern with a general-purpose agent, they trained a model whose entire job is that pattern.

Three things their approach has that mine structurally cannot: probabilities instead of labels, many questions about one state in parallel for nearly nothing, and latency appropriate to something sitting in the hot path.

What I will defend is the integration shape, and it is not about who was first.

Both patterns in their launch writeup lead with the model. Route every request through the classifier. Send every tool call through the classifier. The classifier is the front door.

Mine has three tiers and the classifier is the middle one:

source: "mechanical" | "classifier" | "fallback"

Mechanical runs first and settles genuinely unambiguous cases with no model call. A large share of real traffic is not ambiguous; someone typing an exact pipeline name needs a lookup, not a judgment call. The classifier handles only the residue. A dumb fallback sits underneath so unavailability degrades predictably.

The reason is not elegance. The cheapest call is the one you do not make, and the second cheapest is the one whose answer you can predict without asking. Leading with the model means paying for judgment on inputs that required none.

A second benefit I did not appreciate at first: attributing which tier answered. "The classifier decided this" and "a pattern matched this" are different bugs with different fixes. One undifferentiated tier gives you one undifferentiated failure.

The genuinely new thing in Jev is not asking a model for a typed answer. It is that the classifier finally has purpose-built machinery behind it. What I would push back on is putting it at the front.

One caveat that cuts against adopting it, and worth answering before rather than after: my cost argument rests on no per-call metering, which is exactly what let me put a judgment call where a per-token bill would have discouraged one. A metered classifier reintroduces per-call billing into a system built to avoid it. Probably trivial at my volumes. Probably trivial is a thing to check.

https://openred.space/blog/jev-and-the-classifier-tier.html

Two standards are reviving HTTP 402, and it changes who owns the customer

Michael Hairetis — Wed, 23 Sep 2026 11:06:41 +0000

HTTP 402 has sat unused in the spec since the early web. Two competing standards are now bringing it back, and the reason matters more than the trivia.

x402, from Coinbase. An agent requests a resource, the server answers 402 with payment instructions, the agent signs a stablecoin transaction and retries with proof attached. No account, no login, no stored card. Most traffic settles in USDC on Base or Solana. Coinbase and Cloudflare moved it under a foundation with Circle, Stripe and AWS involved, and it is one of the first extensions to Google's Agent Payments Protocol. It currently carries the most volume of the two.

MPP, the Machine Payments Protocol, from Stripe and Tempo. Spec at mpp.dev, launched March 2026. Two things make it notable. It is not stablecoin-only: Stripe offramps agent stablecoin payments into an ordinary Stripe balance, and fiat and card payments run through it via Shared Payment Tokens, with Visa publishing card specs and an SDK. And it adds a sessions primitive, where an agent authorises a spending limit once and then streams many small payments without settling each one separately.

So the framing is not "stablecoins replace cards." Machine payments need three properties card rails were never designed for: small amounts, high frequency, and authority scoped and revocable per purchase rather than per account. Stablecoin rails got there first because they could. The card networks are arriving through the same protocols rather than being displaced.

The honest state: in March 2026 x402 was processing on the order of tens of thousands of dollars a day, much of it testing, against an ecosystem valued in the billions. The protocol works and the facilitators are live. Widespread use is years away, not a question of whether.

I wrote up what this does to the businesses in the middle. Short version: a delivery app's moat was the interface, not the logistics, and a driver fleet is a hard operational job rather than a defensible asset.

https://openred.space/blog/the-interface-was-the-moat.html

I pointed Google's time-series foundation model at the stock market

Michael Hairetis — Wed, 23 Sep 2026 10:55:56 +0000

I spent two days evaluating Google's TimesFM 3.0 against 118,857 bars of Nasdaq futures tape, strictly causal and session confined. Short version: it is a competent, very small, very cheap component, and it produced no trading edge.

Direction accuracy on price: 50.0%. Exactly fifty. On error magnitude it came in 7.4% worse than assuming the next value equals the last one.

Four framings. Coarser bars did not help. A reframing built deliberately to suit its known strengths did not help. Feeding it side information through its supported covariate channel made forecasts worse at every horizon.

The most interesting failure: I built a probability signal from its own quantile output. When it was most confident it was most wrong. 24.7% hit rate against a 34.3% base rate. Anti-informative when confident is worse than useless, because useless does not tempt you to size up.

The reason is architectural rather than bad luck. It is stateless. Every predict() call sees only the array you hand it, nothing carries between calls, and there is no text interface, so there is no channel through which to tell it anything. It cannot represent "this signal is void while that condition holds." That is a state machine and there is nowhere in a numeric array to put one.

To be fair to it: on a series with stable repeating structure it beat naive by 98.7% zero-shot with 100% direction accuracy. It runs in about 1.3 GB of VRAM, roughly 8% of a mid-range consumer GPU, at about 8 ms per series batched. And on a random walk it did slightly worse than doing nothing, at 47% direction, which is correct behaviour rather than a defect: a model that appeared to forecast a random walk would be inventing structure.

"Foundation" describes how a model was trained and how it transfers. It does not describe competence.

https://michaelhairetis.medium.com/a-foundation-model-is-not-a-foundation-a3edf3383be9

I've had plenty of these so I figure I'd write about them because I'm all of y'all have had at least one by now

Michael Hairetis — Tue, 22 Sep 2026 15:54:30 +0000

Michael Hairetis

Sep 22

The AI Hangover

#ai #hangover #software #programming

3 min read

The AI Hangover

Michael Hairetis — Tue, 22 Sep 2026 15:53:19 +0000

You did not do more work. You did a different kind of work, and it costs more per hour.

There is a state I end some days in now that I did not have two years ago.

It arrives after a long session with a coding agent, and usually not a bad one. The opposite: a session where a great deal got built, the kind of afternoon that used to be a fortnight. At the end of it I am not tired the way a long day of programming used to make me tired. I am flattened. Food does not touch it. Coffee does not touch it. A walk does not touch it. The only thing that resolves it is sleep, and not a nap either.

I have started calling it an AI hangover, and once I named it I found that most people doing this kind of work recognise it immediately and had not said it out loud.

To be clear about what this is: an observation, from one person, repeated enough times to be a pattern. I am not making a claim about neurochemistry and I have not measured anything. What I can do is describe it precisely and argue about the cause, because the obvious explanation is wrong and the wrong explanation leads you to manage it badly.

The obvious explanation, and why it fails
The obvious explanation is volume. You produced four times as much, so of course you are wrecked.

It fails on its own terms. The tiredness does not scale with output. I have had sessions that shipped an enormous amount and left me fine, and sessions that produced one stubborn module and left me unable to form sentences at dinner. If volume were the driver those would be the other way round.

It also fails on kind. Ordinary programming fatigue is a depletion. You feel like you have been running and you slow down gradually, and you can usually feel it coming an hour out. This is not that. This has a distinct quality of overload rather than depletion, it arrives late and abruptly, and the thing that is exhausted is specifically judgment. I can still lift things. I can still hold a conversation. What I cannot do is decide anything.

That last detail is the tell, and it points at the actual cause.

You changed jobs without noticing
Here is what actually changed when you started working this way.

You stopped being an author and became a reviewer.

That sounds like a smaller change than it is, because both activities are called programming and happen at the same desk. They are not the same work, and they do not cost the same.

When you write code yourself, most of the minutes are production. You are typing, and typing is slow, and while your hands are moving your mind is running ahead assembling the next piece. Decisions are real but they are punctuation. They sit between long stretches of execution, and those stretches are where you recover.

When you drive an agent, the production minutes are gone. The agent has them. What is left for you is the part that was always the expensive part: read this, understand what it does, decide whether it is right, decide whether it is right for this codebase, decide whether the thing it did instead of what you asked is better than what you asked, and decide all of that fast enough that you are not the bottleneck.

Every single output is a judgment call, and there is nothing in between them.

You did not take on more work. You took on a stream of work that is nothing but the costly part, with the cheap part removed, and then you did it for six hours because it was going well.

Why reviewing costs more than writing
Two reasons, and both get worse the better the agent gets.

You have to build a model of code you did not write. When you write a function, the mental model arrives for free as a byproduct: you cannot type it without understanding it. When you read a function, the model has to be constructed deliberately, and construction is work. Any developer who has reviewed a large pull request knows that forty minutes of review is worse than forty minutes of writing, and nobody finds that surprising. What is new is doing it all day, at speed, as the entire job.

You cannot fully trust the output, so you never stop watching. Not because agents are bad, but because they are good enough that errors are plausible rather than obvious. A broken thing announces itself. A subtly wrong thing does not, and the only defence is sustained attention. Sustained attention at a constant level, with no natural breaks, is a well-known way to wear a person out, and it is the state you are in from the first prompt to the last.

TimesFM 3.0 is not an LLM, and the word 'foundation' is doing too much work

Michael Hairetis — Mon, 21 Sep 2026 13:44:46 +0000

I evaluated Google's TimesFM 3.0 time-series foundation model on financial data. Before the results, the thing that surprised me most: my own expectations were wrong, and I do not think I was alone.

What I expected from "foundation model"

Memory between calls. Learned behaviour that accumulates. Some way to pass domain knowledge or conditional strategy. The affordances the term carries now that language models have claimed it.

What it actually is

A decoder-only transformer, pre-trained on a broad corpus of time series, used zero-shot. The forecaster exposes three public methods: from_pretrained, predict, predict_batch.

That is the contract. predict() sees a numeric array. It is stateless, so nothing carries between calls. There is no text interface, no instructions, no reasoning step.

Why the distinction matters practically

If your domain model is a set of conditional rules, "signal X is void while regime Y holds", you need somewhere to express conditions. A numeric array is not that place. Those rules are state machines, and they are cheap and exact to write directly in code.

Technically it IS a foundation model: pre-trained once, applied zero-shot across domains with no per-series fitting. That is genuine, and the deployment numbers back it up. But "foundation" describes how it was trained and how it transfers, not what it can do.

Full series, including why it produced no usable signal on market data:

https://michaelhairetis.medium.com/i-expected-memory-i-got-a-function-call-40b714cac7b6

You cannot make the agent fast enough. Move the wait instead.

Michael Hairetis — Fri, 18 Sep 2026 12:26:32 +0000

I built a small learning app for my two kids on the same agent infrastructure that runs two other platforms. It introduced a constraint the other two never had.

Somebody is waiting.

The orchestration platform runs at four in the morning with nobody watching. If a call takes ninety seconds, it takes ninety seconds, and nothing in the system notices. The publishing platform has a person in front of it, but that person is me, reviewing on my own schedule.

The third has a second-grader holding a pencil, looking at a screen. Her patience is not a performance budget I get to negotiate. It is a hard physical limit, and when I exceed it she wanders off and the product has failed in the only way that matters.

I built it the obvious way first, and the obvious way was wrong.

Generate content one question at a time. Each call is small, so it is fast and unlikely to come back malformed, and I prefetch the next question while the child works on the current one. In theory the waiting hides behind the thinking.

In practice a child does not fill the gap the way a scheduler does. They answer, and then they sit. If the prefetch had not landed they watched a spinner, and they did that between every single question. Six items in a set, six opportunities to lose them.

The architecture that looked responsive on paper was the least responsive thing I could have built, because it distributed the waiting across exactly the moments when the user had nothing to do.

The instinct is to make the calls faster. That instinct is wrong.

You usually cannot make an agent fast enough to be invisible, so stop trying and change when it runs instead.

The app now plans three lessons at once and builds each one complete in a single call, every question and figure and answer finished and stored before the child sees any of it. Lesson one takes a minute or two. They wait for that, once. While they work through it, lessons two and three build in the background. From there, opening a lesson is a database read. I measured it at six milliseconds.

The waiting did not shrink. It moved. One wait at the front, where a user will tolerate it because nothing has started yet, in exchange for zero waiting during the part where they are actually engaged.

Two details make it hold up. The lesson has to be genuinely complete, not a plan with placeholders, or the scheme collapses back into the original problem. And the background build has to survive a server restart, because in-flight async tasks die silently and leave a lesson stuck at "building" forever.

The full piece covers the applets the agent writes as working code, the positional bias I found in generated quizzes, and why my reader cannot read.

https://openred.space/blog/a-seven-year-old-will-not-wait.html

Does your system read back what it writes?

Michael Hairetis — Wed, 16 Sep 2026 13:00:20 +0000

I had agents writing notes that nothing ever read. For months. Here is how I found out, and why I think it is a common shape of bug rather than a one-off.

The setup. An orchestration platform runs scheduled data jobs. Worker agents were instructed to bank durable technique in a per-job notes file: parsing traps, cadence, known-good access patterns. They complied. Some of those files reached 40 KB.

The check that took thirty seconds. I grepped the codebase for the filename.

Two hits. An archiving routine, and a UI file-lister.

It was never injected into a prompt. Never read at plan time, never read at execution time. Write-only memory.

The tell I had already seen and misread. Only a fraction of jobs maintained a notes file at all. I had assumed inconsistent agent behaviour. It was rational behaviour: nobody keeps a notebook that never comes back to them, and that turns out to be true of software too.

The second half of the same bug. The plan handed to each worker was discarded when the run ended. So a job that had run seventy times re-derived its plan from zero every morning. Schema, paths, identity rules, all rebuilt nightly from nothing.

Measured cost: a clean run needs five agent turns. Successful runs averaged 6.5, worst case 10. Runs that concluded "nothing new today" averaged 3.5, burning turns after the answer was already known.

Three constraints worth stealing if you wire this up.

Bounded, not whole. My largest notes file was nearly double the size at which my prompt transport starts corrupting input. Injecting it wholesale would have reproduced a bug I had already fixed. The file now has a curated head that travels with every run and a tail that stays on disk at a path the agent knows.

Curated, not appended. Append-only is how you get bloat. Replace facts that changed, delete the superseded version, and if nothing durable changed, leave it alone. That last clause prevents churn.

Instructions survive, context yields. When plan plus notes would exceed the ceiling, the notes get trimmed and the plan never does.

And one prompt-level detail that mattered more than the architecture: tell the planner explicitly that "nothing new today" still means the plan worked. Without that, it reads a quiet outcome as underperformance and rewrites a plan that was fine.

Go grep for the filename your agents write to. It takes thirty seconds and I would bet on the result.

https://openred.space/blog/agents-have-amnesia.html

Every editorial bug I shipped was an agent following orders exactly

Michael Hairetis — Tue, 15 Sep 2026 14:54:19 +0000

I run two platforms on the same agent infrastructure. One makes internal decisions, the other publishes prose to actual readers. The second one taught me something the first could not.

For about a month I kept finding defects in published text. Internal shorthand in a reader-facing card. Database row identifiers appearing in a published sentence, literally events/1042 in a paragraph a subscriber could read. Terms of art used with no gloss. A section quietly editorialising instead of reporting.

Every single time, my first instinct was that the agent had drifted, misunderstood, or needed a firmer instruction.

Every single time, I was wrong.

The agents were following their role definitions to the letter. The problem was that each definition had been written for an internal audience and then quietly promoted to a reader-facing one.

The clearest case: the public writer had an instruction to faithfully preserve the source's framing. Perfectly reasonable. But the sources were Treasury and Federal Reserve releases, and they use the internal vocabulary. So the agent was being obedient when it passed that vocabulary straight through to readers.

It was not drifting. It was following an order that had become wrong the moment its output started being published.

Another role had an explicit invariant instructing it to cite records by identifier rather than by name, because that was precise and useful when its only reader was me. When its output was later routed into a published summary, that same invariant produced events/1042 in the prose. The instruction never changed. The audience did.

So the debugging question is not "what did the agent get wrong."

It is: who does this role think it is writing for, and is that still who reads it?

I now treat every role definition as an editorial brief with a named audience. When a role's output changes destination, the definition gets rewritten rather than patched.

The failure mode nobody warns you about is not the agent going off-script. It is the agent following a script you wrote for a different reader and forgot to update.

The full piece covers what happens when three agents read the same document with different jobs, why I deleted my entire document-extraction layer, and how agent failures needed a taxonomy rather than a retry.

https://openred.space/blog/calling-an-agent-for-sentences.html