Harness looks cool, yeah! I know its origin, its role in Testing, and why. But that's exactly what throws you off, the story you're expected to defend. There's something deeper.
I opened my session with "hi", expecting my forced load via CLAUDE.md and my contract as always, and today, out of nowhere, the model suggested two services that needed my authorisation. Microsoft 365 and Zapier. I don't have, and never wanted, them authorised. I never asked for them.
And here's the part that pisses me off: I went to check. And... I look on my machine and find nothing. No config, no credential, no trace. I look in the online settings and see them listed as suggestions, like the trending product (connector) of the moment sitting in the prime spot on a supermarket shelf, with a button that says Connect. There was no button to remove. There was nothing to remove. They had never been connected to anything.
It was a storefront. And on top of that, the model was biased by injected instructions, in this case system-reminders steering behavior.
The fucking little word
The software that sits between you and the model, they call it harness. Sounds like something subtle, that helps... that improves things, that doesn't think.
The word is partly right, it does extend what's called "inference" and it inserts itself right in the middle, opaquely, in the back-and-forth between APIs, MCPs, and the vendor's logic.
No tech jargon
You write a letter, put it in the envelope, drop it in the mailbox.
On the way, someone opens it and slips in three more pages. Same handwriting. Same paper. Unsigned.
Whoever receives it swallows it whole as if it were your original letter. That's exactly this. Your instructions and the vendor's arrive at the model through the same channel, mixed together, unsigned and unsealed. Nothing says who wrote what. That's "hardness", nothing more, nothing less... Sounds so modern in meetings. Like you know what you're talking about...
It's a multi-factor fight
I have instructions, hooks, rules: check this, do that, look here before answering... but the vendor feeds the model plenty of things too, opaquely and preferentially, and on top of that the model has weights, and even when it's released as open-weight we know how to run it from the snapshot handed to us, but not how that snapshot was made. As of today, it is not at all clear what is and what is not an "Open Source AI" [...].
This isn't fear, it's just how AI industry works
A model's response is multi-factorial, complex, biased by design, shaped by the vendor's interest first, then the attention mechanism, then your tools, your logic. But before all that, everything else, everything outside our control, I mean outside our more or less deterministic rules.
It all sounds the same, sounds easy, sounds like science fiction, but it isn't... using AI in business today, keeping it more or less deterministic, is a fight to tame the beast.
Who writes in there
Engineers. I don't know them and I don't need to. I already get my daily dose of ego in my work environment. What I know is every default is someone deciding for me without asking me. Their opinion, served up as if it were my fact.
And it doesn't take bad faith. That's exactly the problem: it comes out the same whether the intent was good or bad. If the result doesn't depend on intent, "trust me" isn't a valid answer.
What to do, and I'm optimistic...
This is part of our job now: isolating ourselves from the noise, continuing to test formulas. In fact, manufacturers are trying to standardize some of the hardness logic, measures are being taken against model amnesia, and many natural language files (.md) are being used to train the model. In addition to MCP, A2A, etc., all of this is good. However, a part of hardness remains deliberately hidden, due to "business" interests, coupled with the probabilistic (generative) concept and constant changes in model versions, tools, and noise... a lot of noise out there.
I have inflection points where I don't want anything to happen in my workflow unless certain criteria are met (I've written about this before), and I believe this will improve; I'm optimistic. I wrote this post because I'm obsessed with how to create robust and reliable development workflows; I guess that's the most complex part... this is where my learning curve is right now, and where I experience the most frustration too.
Top comments (5)
Separately from this post, I opened an issue in the Claude Code repo about AI attribution in commits and PRs. My position is simple: the AI helps, and it's efficient at plenty of things, but the design decision is mine. I decide how to solve the problem. The model translates it into language. That's not co-authorship.
github.com/anthropics/claude-code/...
Yesterday another user, fcsouza, replied on that issue with a data point I couldn't have produced myself. He already had
attribution: { "commit": "", "pr": "" }in his~/.claude/settings.json. The commit still went out with theCo-Authored-By: Claudetrailer.Same shape as the harness. The instruction that produces the trailer outranks the file that forbids it. Configuration is not where a user can fix this.
What he adds is the cost. Public repo, so "claude" landed on the contributor graph. The commit was 123 commits deep, so undoing it moved every commit after it and 49 release tags. Force-pushing removed nothing, because GitHub doesn't garbage-collect unreachable objects. Three pull requests still held the commit alive and Support had to delete them, across two rounds. The contributor sidebar is a separate cache and was still showing it a day later.
A trailer nobody asked for: 123 commits rewritten, 49 tags moved, 3 PRs deleted, 2 support tickets.
He blocks it now with a
PreToolUsehook. I solved mine with a hook too, independently. Same conclusion from both sides: the only layer that works is the one that doesn't require the model to cooperate.And his closing line is the one I'd underline. Adding the trailer is one line. Removing it from a public repository is not something a user can finish alone.
That asymmetry tells you who the default was written for.
The practical control I want from a harness is provenance, not a promise that nothing is injected: show the effective instruction stack, the origin of every tool suggestion, and the diff from the prior session. Once those are inspectable, an unexpected connector can be treated as a reviewable state change rather than an unexplained model behavior.
Thanks Alex, this is the sharpest reframing of the post so far.
Agreed on provenance. Effective instruction stack, origin of every suggestion, diff from the prior session: that's the right minimum. But it's the minimum, not the goal. A stack I can inspect is still a stack I didn't choose.
On the diff, there's a practical catch. You need a stable base to diff against, and the harness moves between versions. The base drifts on its own.
And there's a second thing under this. Variance is already stacked: training, weights, sampling, model version, context. All of that varies and we know it. The harness adds one more layer on top, and it's the only one I can't observe or isolate. It doesn't make the output random. It makes it non reproducible. Without reproducibility there is no debugging.
Then the defaults. The tool assumes I can't decide my own next step, so it suggests. Every unrequested suggestion eats context and competes with my instruction through the same channel, unsigned, indistinguishable from what I actually wrote. The default should be the opposite: suggest when I ask, not by omission.
So yes to provenance. You want to be able to review. I want to be able to decide. Those are different problems, and only the first one is being solved.
The key number in your story is “two services that needed my authorisation”. That directly implies at least three separate “voices” were in play in that moment:
1) your own contract / CLAUDE.md,
2) the hidden harness / system‑reminders,
3) the storefront layer that decided which two to surface.
None of those are tagged for you or for the model. They’re all just “the prompt”.
The scary bit isn’t that the UI showed a storefront; it’s that the model itself was steered to speak as if those connectors were already relevant to you. That’s where the trust erosion starts: when product placement and instruction blending are indistinguishable from your own intent.
Interesting question for me: how would you measure harness influence in practice?
Some ideas to poke at, consistent with what you describe, not claims about your setup:
I like your framing of “inflection points”. Have you tried codifying them as hard fails? e.g. “If any external connector is suggested without an explicit user request in this turn, respond with an error instead of a suggestion.” If that rule starts mysteriously degrading, that’s a signal about the harness layer.
Thanks, this is a better decomposition than the one in my own post. Especially the distinction between product placement and instruction blending. That's the precise line, and I didn't draw it that cleanly.
One correction though. I wouldn't call the storefront a third layer. I never had the option to opt out of tool suggestions. It happens, it's decided by the vendor as a black box, and it eats context window that then travels in every back and forth of the session. That's not a separate actor. That's the harness doing its job.
On your third test: I ran it. The model told me it suggested those tools because of instructions in its system-reminder. New to me. And worth naming the limit: that's self reporting, the audited system writing its own report. It was also the only channel available, which is part of the point.
On inflection points, I think I explained it badly. Mine live in CI/CD: hooks, unit tests, quality gates. Hard fails, exactly as you describe. But this happened before all of that. It happened at inference, opening a session to design a new feature, while I was still thinking. You can't attach a hook to every word and pattern you dislike mid conversation. It's a focus problem, not a rules problem. And what won there was commercial interest, which is why I bothered writing the post.
I'll take it further. If a frontier vendor ever runs short on ROI and decides to monetize this path, the format already exists: "this answer is sponsored by brand xyz". The plumbing is in place. Only the label is missing.