DEV Community: Pierre- Laurent Medori

What does a JWT actually prove? Less than your API assumes

Pierre- Laurent Medori — Thu, 24 Sep 2026 12:12:00 +0000

View source on one of our generated widgets and you can find a tenant id, a connector handle and the address of a token issuer. Take those values to a terminal and you can obtain a valid token. Nobody has stolen a signing key. The issuer is doing what it was built to do.

I run backend engineering at GoodBarber, where we are testing an AI extension builder. Its widgets call third-party APIs through a proxy that holds the real API keys. The feature is a prototype serving a handful of pilot tenants; the design below is still evolving.

The token tells the proxy which tenant the issuer accepted a request for. It does not identify the person making the call. That distinction determines what we can safely authorize, and it leaves a residual risk that a signature cannot remove.

What a bearer token proves

A bearer token grants its permissions to whoever possesses it. Its use does not require a separate proof that the presenter holds a cryptographic key. That is the model described by RFC 6750, section 1.2.

We use signed JWTs. Verifying one against a trusted issuer's key establishes that the signed claims have not been altered. The consumer must also check the claims required by its protocol, including issuer, audience and expiry. JWT itself supports other forms, including encryption, and does not require every application to use the same claims.

For our bearer tokens, verification does not establish who is holding the token now. Nor does a signature make an unverified claim true. If an issuer copies a caller-supplied label into sub without authenticating that caller, the downstream API cannot turn it into an authenticated identity by checking the signature.

What our issuer checks before it signs

Our issuer mints tokens for a tenant id. The endpoint is public. Before it signs, it asks the client to solve a small proof-of-work, which makes minting cost CPU rather than a login. The proof-of-work is a cost, not a gate: a script that is willing to spend the cycles gets a token. The issuer checks that the tenant is a legitimate customer when it creates that tenant's signing key; after that, every request for that tenant is served within the rate limits.

Each tenant has its own RS256 key pair, the public half is published as a JWKS per tenant, and tokens live fifteen minutes. After verification, the proxy has a tenant label backed by our minting process: somebody completed its challenge for that tenant recently enough for the token to remain valid. That somebody may be an untrusted caller.

The question we stopped asking

We first considered a signed context from the native app: something the mobile runtime could attach to a request, independently of the page. Our production SDK does not provide that, and the web version of the widget has no native runtime to ask.

So we designed for the caller who can obtain a valid token without earning our trust. The proxy has to enforce three constraints even for that caller.

The secret never leaves the server. The third-party key exists in plaintext only in the proxy's memory, for the duration of one outbound request. It is not in the page, not in the bundle, not in the response, not in the logs.
Egress is bounded by what was provisioned, not by what the caller asks. A connector is registered by an authenticated developer with a host, a base path and a list of verbs. The widget sends a relative path. It never sends a host.
Abuse is capped. Per-tenant and per-connector rate limits, a kill switch per connector, and a global one.

The order of operations follows from the first invariant. In pseudo-code, one request through the proxy looks like this:

verify token             -> 401 if invalid; 503 if verification keys are unavailable
resolve connector        -> by (tenant from token, handle from URL), no secret loaded
check kill switch        -> 503
check verb allowlist     -> 403
check path allowlist     -> 403
parse destination        -> host must match the provisioned allowlist
resolve DNS and pin IP   -> block private ranges, keep Host and SNI for TLS
count against limits     -> 429
decrypt the secret       -> only now
inject, forward, redact  -> scrub the key and the credential forms our injectors construct

Those admission checks run before decryption. On the way back, the response is scrubbed against the key and the credential forms the injectors construct, including a Bearer header or a base64 Basic pair.

A caller with a valid token can still invoke a provisioned connector, within its permitted host, path and verbs, until the quota says stop. The controls are intended to prevent extracting the key or redirecting its use beyond that scope. They do not prevent every harmful use within it. Burning someone else's API quota is one such use.

What the token still buys

The issuer still has useful work to do.

Attribution and cost. Every request carries a tenant id that the caller had to spend something to obtain, and every abuse has a label. Rate limits per tenant only make sense when the tenant claim cannot be free-typed by the caller. The signature is what makes the label trustworthy, even though the label is not an identity.

Browser restrictions. The mint runs inside an iframe served by the issuer. Its allowed parent origins come from our tenant registry, and the proxy applies the same per-tenant list to CORS. An unrelated site cannot simply embed that mint in its own page. A client outside the browser remains a different case.

The native limit. Our iOS plugin pages share one fixed origin across apps. Android's asset-loader origin is shared by other apps using that loader too. Neither origin distinguishes one app installation from another. The token's minting cost and the proxy's quotas still apply, but an origin check adds no installation identity.

A possible extension. DPoP, specified in RFC 9449, can bind an access token to a client-held key and require a proof with each request. Adopting it would require validation of that proof and its binding to the token at the resource server. It would add a possession check; identifying the person behind the key would remain a separate question.

Writing the residual risk down

Before writing the proxy, we put the remaining risk into a paragraph of the security specification and recorded where we accepted it.

The paragraph says, in substance: anyone who can mint a token for a tenant and knows a connector handle, which is public in the bundle, can invoke that connector within its declared scope, burn quota or trigger the side effects the connector allows, without ever extracting the key. It lists the five things that bound that risk: the secret is not exfiltrable, the allowlist is per credential, the limits, the kill switch, and the upstream control over who can publish code into a tenant's app. It says the risk is accepted for read-only connectors and that side-effecting ones need tighter quotas and idempotency.

That paragraph gave reviewers specific questions to investigate: could a redirect expose the secret? An error page? A log line or a cache? Several findings produced fixes. The specification grew to 108 numbered requirements, each phrased so that a test can fail it. The risk paragraph explains why those requirements exist and which harms can remain when they all hold.

What we would change

Three changes remain on our list:

A dedicated audience for each consumer of the issuer, before a second consumer exists. A token meant for one service should not verify on another.
Per-tenant quota overrides on the proxy, before opening the feature beyond the current pilot tenants. Aggregate limits per connector are the right posture against abuse and the wrong posture for a popular app.
A signed install context, the day the native SDK can provide one. The proxy will accept it as an additional check, not as a replacement for the invariants.

On September 16 I verified the same published widget on the web, on an iOS simulator and on an Android emulator: one GET and one POST through the proxy, the response redacted, the key absent from the page. Those checks exercised the request path. The adversarial reviews examined specific ways to escape it. Neither is a proof that no other attack exists.

An API using this design has to be willing to grant the connector's declared capabilities to someone who can obtain the public token. If a capability requires an identified user, that identity needs its own authentication step. Quotas and a signature cannot supply it.

Before trusting a claim downstream, I want to know which check caused the issuer to write it. Which claim in your API would be hardest to trace back to that check?

How do you stop an LLM from leaking API keys in the code it writes? Default to secret

Pierre- Laurent Medori — Tue, 22 Sep 2026 12:00:55 +0000

On July 22 I typed a one-line prompt on a test app: a widget built on a well-known REST API, the kind that authenticates with an X-Api-Key header. The model wrote a clean widget. It also wrote my key into a request header, in the page's JavaScript. In the declaration that comes with the code, it had marked the API as not secret.

It was my own key, on my own test app. At GoodBarber we are piloting a builder in which an app owner describes an extension and a model generates its code. Nothing had been released yet, and a review before release is the cheapest place to find a key in the wrong spot.

What we changed is the answer to the question in the title. We do not scan generated code for secrets, and we do not let the model decide whether a key is secret. The model declares every key an API needs and states one fact about it: how the key is sent. Code does the classification: a key is secret unless it belongs to a family its provider documents as public. The value goes from a masked field to a server-side proxy. It never reaches the model, the page or the bundle.

A key that looked public

The model was not being careless. The keys it treated as secret without being told were the famous ones, AI providers and payment secret keys. A forty-character string with no prefix, sent in a custom header to a REST API, looks like what front-end tutorials do all day. And that reading is not absurd, because some keys really are public.

I have written on the company blog that security is the best-documented wall between a demo and production. This is what that wall looks like from the inside, on a Wednesday.

Some keys are public by design

Stripe lists its publishable key as safe to expose. Firebase documents its API keys as public by design. Supabase documents its publishable key as safe to expose online, and Algolia its search-only key as safe for production front-end code. These keys identify a project. Authorization happens elsewhere: security rules, row-level policies, restricted scopes.

So "never put a key in the page" is wrong, and a platform that enforced it would break every one of those integrations. The real question is classification. A model will get that call right many times and wrong some times, and one wrong call publishes a key.

Why we did not scan the code

The reflex is to search the generated JavaScript for things that look like keys. Two things argued against it.

We had already lived through pattern-matching on generated code. An earlier validator matched unsafe JavaScript constructs with a list of regular expressions. It produced enough false positives to be switched off, and I deleted it on July 16. The design notes for the secrets work say it in one line: the secret guard must not reintroduce pattern-matching on generated code.

And most secrets do not look like anything. GitHub redesigned its token format in 2021 because its old tokens were hard to tell from a SHA hash; with identifiable prefixes, it expected the false positive rate of secret scanning to fall to 0.5 %. That works when the provider cooperates. GitGuardian's 2025 report counts 23,770,171 new hardcoded secrets in public GitHub commits in 2024, and classifies 58 % of the leaks it detected as generic credentials, the ones no provider-specific pattern describes. The key in my widget was one of those. Forty characters, no prefix.

A scanner would have been blind to the exact key it was built for, and noisy about everything else.

A decision with a tripwire

Here is the part I am glad we wrote down. In the plan, dated July 13, the model's secret flag was trusted, and a stricter guard, secret unless proven public, was considered and deferred. Deferred with a condition, in writing: revisit if a leak by under-classification is observed.

It was observed on July 22, on my test app. The stricter guard was committed the same afternoon, at 15:01.

A deferred safeguard without a written trigger is a safeguard that "later" never schedules. The trigger turned a debate into a lookup.

Default to secret, decided by code

The fix has two layers, and neither reads generated code. Both read the declaration: the structured list of APIs and keys that the model must produce alongside the widget.

The first layer is the instruction given to the model. Its header says "DEFAULT TO SECRET when unsure", and its core is one rule: a key is secret unless the provider explicitly documents it as public, publishable, anon, browser or search-only, and a key that authenticates your account is secret no matter how it is sent, a custom header and a query parameter included.

The second layer is the one that matters, because an instruction is a request. A validator checks the declaration. An API declared as not secret, whose key does not belong to a family that providers document as browser-safe, is rejected. The rejection feeds the retry loop we already had, with a message that tells the model to route the call through the proxy helper. The exemption reads the structured fields of the declaration, never its prose: a description saying "get it from the public dashboard" must not make a key public.

The model can still raise a key to secret. It can no longer lower one. Only evidence can.

Ask the model for facts, not for judgment

The next morning showed that the rule was not enough. Reclassifying a key as secret is useless if nobody knows how to send it. To wire a server-side connector, the platform needs to know whether the key travels as a bearer token, a named header, a query parameter or a path segment. The model had been asked for that only when it judged the key secret. A key it judged public arrived with a name and a label, and nothing to build a connector from.

The instruction now asks for the transport of every key, always, whatever the model thinks of its secrecy. The prompt says why in its own words: it is "a FACTUAL property of the API, not a consequence of the secret-vs-public call".

This is the distinction I would keep if I had to drop everything else. Where does this API expect its key, a header called X-Api-Key, a query parameter called appid? That is documented, and models are good at documented facts. Is this key dangerous in a browser? That is a judgment about consequences, and the model's prior is a web full of inlined keys. So the contract asks for the fact unconditionally, and code makes the judgment.

The same part of the prompt carries the inverse rule: never invent a fact. When an API authenticates with an OAuth2 token exchange or per-request signing, none of the declared transports describes it. The model is told to say so plainly and leave the API out, rather than declare a static bearer token that would pass validation, collect a key, and fail with a 401 on every call.

The same rule at every door

One more place needed the rule. Our generation runs in two phases: a plan the owner approves, then the code. The key is requested when the plan is approved, before any code exists. A guard that runs on generated code arrives too late, because the question "is this a secret?" has already been answered by then. The classification now runs at the single point where all four completion paths collect what must be asked of the owner.

A classification rule has to live at every boundary the value can cross. Otherwise the earliest boundary wins.

There is a second net where the value itself enters, the consent form and the credentials screen. If a pasted value carries an unambiguous vendor signature, sk-, ghp_, xoxb-, AKIA, a PEM header, it is routed to the proxy even when its declaration said public. That net is deliberately narrow, because it must never match a publishable key. Ambiguity is handled upstream, by the default.

Twenty-five out of fifty-one

A default that leans towards "secret" can overcorrect. Two weeks later one of ours did, in a neighbouring rule, for one day. It is the part of this story I find most useful, because it is the opposite failure.

Some APIs require a non-secret header on every call: a version header for Anthropic or Notion, a host header for RapidAPI. We let the model declare those as constants. Constants are stored in clear, so a constant must never carry credential material, and the failure to guard against is the model splitting a pair like X-RapidAPI-Host and X-RapidAPI-Key the wrong way round.

The first guard, written on August 4, refused any constant whose name contained one of ten markers, key, token, secret and auth among them. It looked strict and safe. The next day a review pass ran it against 51 header names taken from real API documentation, and it refused 25 of them. X-Auth-Email, which Cloudflare pairs with X-Auth-Key, is an e-mail address. X-Auth-Client, X-Auth-User, X-Oauth-Version, Authority: all legitimate, all refused.

A false positive here is not a harmless excess of caution. A refused constant means a rejected declaration, a retry loop that burns its attempts, and an API the owner legitimately wants that cannot be integrated at all. The same day, a sibling rule turned out to forbid Authorization as the place where a key is injected, which is exactly where some APIs expect it. Two over-blocks in one family, fixed the same morning.

In the names we tested, the last segment carried the meaning. X-Auth-Key is a secret. X-Auth-Email is not. The guard now splits the name on its separators and reads the last segment only, against a slightly wider list of markers. Rerun the same day: 21 declaration patterns from real APIs accepted, then 12 more, and 7 out of 7 secret-carrying names still refused.

That is an observation about those families of headers, not a law of HTTP. Idempotency-Key ends in key and is not a secret. As a constant it is still refused, and I can live with that: an idempotency key changes with every request and has no business being a constant. A heuristic earns its place by being measured on real names, in both directions.

Twenty-five out of fifty-one. A guard that refuses half of the legitimate world is not strict. It is wrong in the other direction, and people route around guards that are wrong.

Choose which way to be wrong

The errors now fall on the cheap side. A public key classified as secret costs one proxied call instead of a direct one: slower, and no key is disclosed. A secret key classified as public costs the key. When a classifier has to be wrong sometimes, choose which way, then measure the other direction too.

If a model writes code that calls paid APIs on behalf of your users, where does the decision "this key is secret" live in your system: in a prompt, in a scanner, or in code that can say no?

Listing a remote MCP server in every directory: what each one actually checks (September 2026)

Pierre- Laurent Medori — Thu, 17 Sep 2026 11:52:00 +0000

The Save button did nothing. We opened the network panel. The request had returned HTTP 200.

Inside the response: a validation error on a field the form did not show.

By then we had spent several days trying to get the same remote MCP server listed across directories. Another directory had published us in about ten minutes. A GitHub list had tested our endpoint and accepted its 401. Elsewhere, our submission was waiting for a human.

"Get the server listed" had turned into a collection of different jobs.

One line of context so you know where I stand: I run engineering at GoodBarber, an app platform, and our MCP server lets assistants manage the apps through an authenticated connection. It is hosted, it requires OAuth, and there is no public server code for a directory to scan: our public repository holds skills and client configuration. That distinction decided which submission paths worked.

After getting the server into ChatGPT's plugin directory, we widened the search. What follows happened between September 7 and 14, and every public state was checked again on September 14. The delays are the ones our submissions went through, not a promise from anyone.

The official MCP Registry: publish the identity first

The official MCP Registry accepts metadata for hosted servers, closed-source implementations included, as long as the server is publicly reachable. It authenticates the publisher's namespace and validates the metadata. Beyond that, its metadata is deliberately unopinionated, and curation is left to the aggregators downstream.

Our entry, dev.goodbarber/goodbarber-public-mcp, has been there since April 28 (version 1.0.0). On September 7 we published version 1.2.1, with DNS authentication: it declares streamable-http, the remote endpoint and the product page. On September 14 the registry API still marked it active and latest.

We would do this first again. It gives importers a stable identity and a record to consume, instead of asking each directory to rebuild the product from a README.

Glama had already imported a connector entry from that record. We claimed it, and on September 8 it displayed "Ownership verified" and "Healthy". Both were still there on September 14.

Upstream publication does not tell you when the downstream pages will exist. MCP.Directory says it auto-discovers servers from the official registry, where our entry has been since April. Its submission form promises a review within 24 hours, and its first, required field is a GitHub repository, from which it detects the tools by analyzing the server's implementation. The Knights Who Say Ni wanted a shrubbery; a hosted server with no public code has no repository to give. On September 14, MCP.Directory's sitemap still had no GoodBarber URL, six days after a server listing request and five skill submissions. Where those submissions sit, we do not know.

awesome-remote-mcp-servers: the CI that accepts a 401

awesome-remote-mcp-servers is built for our shape of product. It only lists servers the provider hosts: reachable at a public URL, usable by anyone who can sign up, speaking Streamable HTTP or SSE. Each entry's name links to the product's homepage, not to a GitHub repository, because the endpoint is the thing being listed. The list was created on September 8, and our PR went in the same morning.

It also gave us something we could test. Its submission workflow at the merge commit probes each endpoint with a real initialize handshake and sends no credentials. We replayed the check before submitting. Our endpoint answered with an authentication challenge, structurally like this (generic domain):

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Bearer resource_metadata="https://mcp.example.com/.well-known/oauth-protected-resource"

The workflow takes a 401 or a 403 as a live endpoint that requires authentication, and reads the WWW-Authenticate header to choose between the OAuth and API-key markers. Our 401 passed with the OAuth marker. The probe never completes OAuth and never calls a tool: a green check here means the door exists.

Two more rules matter before you open the PR. Every entry needs a Glama connector badge, and CI checks that the connector exists. And the contribution rules document a fast track for automated agents: three robot emoji at the end of the PR title. Our PR was prepared by an agent, so we used it. PR #4 was opened at 07:23 UTC, labeled endpoint-ok and has-connector by CI nine seconds later, and merged at 13:50 UTC, on September 8.

Then the bot ran again on the merged PR. It added duplicate and missing-connector, removed has-connector, and asked us to remove the duplicate entry. The duplicate was ours: the check was now reading the list it had just merged. The README on the main branch settled what the labels could not.

Gemini CLI and GitHub: a crawler versus a review queue

The Gemini CLI gallery instructions describe a daily crawl of public GitHub repositories carrying the gemini-cli-extension topic. A gemini-extension.json manifest at the root of the repository supplies the metadata, and an extension appears in the gallery only if it passes validation. The instructions say there is no issue to file and no email to send.

We added the topic on September 9. On September 14 the topic and the manifest were in place, and the gallery page had no GoodBarber entry. A daily crawl gives no per-repository result, so we cannot tell whether we failed validation or were never picked up.

GitHub's MCP gallery has another gate. In discussion #1257, a GitHub answer explains that onboarding a new server is a manual curation process, and that new registry versions sync once a server has been onboarded. We asked to be included on September 8; on September 14 the expected gallery URL still returned 404. Republishing the registry record would change nothing there.

Our Claude Code plugin sits in a similar waiting room. Plugins are submitted through a form, and the public community marketplace catalogue is a read-only mirror, synced nightly from Anthropic's review pipeline, of the plugins that passed automated security scanning and were approved. Ours was submitted on September 7, shown as awaiting review on September 8, and still absent from that catalogue on September 14.

Claude connectors: ten minutes, then an inventory problem

The Claude connectors directory is a different destination from the Claude Code plugin marketplace. Remote servers are submitted from an organization's admin settings, through a portal that connects to the server. Our connector listing appeared about ten minutes after we submitted it on September 8, while the plugin was still waiting. Its tier is community, and that is the word we use.

Then we read the published inventory and found our own mistake. The directory's public record listed 120 tool names and left out an entire tool family, the one behind push notifications and analytics, while the description we wrote advertised both. The portal syncs the tool list from the server through the connection made at submission, and our connection did not expose that family. The description is ours to edit from the dashboard; the tool list is not an editable field, and changing it goes through the review team. Either way, the fix is ours.

For someone following the Claude connection guide, the only question that matters is whether the tools they get cover the task they came for. A directory listing does not change the tools a connection exposes, so the check we owe that reader goes beyond the listing: the published inventory against the description, then a real connection.

mcp.so: HTTP 200, application-level failure

The Save button from the opening was on mcp.so. We were updating an existing listing whose transport and overview had gone stale.

On September 8 the edit request returned HTTP 200, and its body carried application code -1, an "Invalid submission" message and a validation error under fieldErrors.tagline. The old tagline was too long. The form had no field to fix it and showed no error.

We sent the edit again with a shorter tagline, then read the public page, because a save response is a claim, not a result. On September 14 the listing showed Streamable HTTP, OAuth and the updated overview.

If that sounds familiar, it is the write-safety piece in miniature, with a web form in place of an agent. The green request in the network panel meant an HTTP exchange had completed, nothing more. Read the body, then read the page.

MCP Market: fast, paid, and one button to check

On September 8 we paid $69 for the remote-server "Official" listing option, which is a different product from the GitHub-repository submission. The listing was live the next morning, inside the advertised 24 hours, with the badge and the categories we asked for.

Search MCP Market for GoodBarber and a second card shows up next to ours: GoodBarber Skills, filed under Marketing Automation.

Then we clicked Try Now.

We had submitted our MCP product page as the destination. The published button led to our app-creation page, with MCP Market campaign parameters added, and still did on September 14. Both pages are ours, and it matters anyway: someone looking for connection instructions lands on a different step of the product journey.

Paying got the listing published fast. Checking where it sends people stayed our job.

Smithery and skills.sh: read what the signal measures

On Smithery, verification happens in the server's settings, through what its publishing guide calls an automatic official-vendor verification checklist. Our domain and backlink proofs passed their recheck on September 8. The public data embedded in the listing page said verified: false afterwards, and still did on September 14. The guide names the checklist without listing its items, and nothing we could see explains the step between two passed proofs and a false flag. We write down both states rather than calling the whole thing verified.

skills.sh showed our 44 skills on September 14, with installation counts. There is nothing to submit there: skills appear through the anonymous install telemetry of its CLI, when people run npx skills add. Those skills are instruction packages for using the server, and the counter measures installs: not a completed workflow, and not an OAuth flow that works in every client.

The September 14 snapshot

Every state below was checked on September 14, 2026; submission and publication dates are the real ones. Read "pending" as pending, and "absent" as unexplained.

Directory or catalogue	Admission mechanism	Observed delay	State on September 14	Pitfall
Official MCP Registry	Namespace authentication, metadata validation	Entry since April 28; 1.2.1 on September 7	Version 1.2.1 active and latest	Curation is left to aggregators
Glama	Registry import, ownership claim, health checks	Initial import not timed	Ownership verified; Healthy	Ownership and health are separate signals
awesome-remote-mcp-servers	Endpoint CI without credentials, Glama connector badge, PR	Same day, September 8	PR #4 merged	The probe stops at the 401
MCP.Directory	Registry auto-discovery; GitHub-based form with a 24-hour review promise	September 8 to 14, unresolved	No GoodBarber URL in sitemap	The form expects a GitHub repository
Gemini CLI gallery	Topic crawl, root manifest, validation	September 9 to 14, unresolved	No entry on the gallery page	A daily crawl gives no per-repo result
GitHub MCP gallery	Manual curation, then version sync	September 8 to 14, unresolved	Expected URL returns 404	Registry publication does not grant admission
Claude Code community marketplace	Submission form, security scan, approval, nightly mirror	September 7 to 14, unpublished	Absent from public catalogue	Separate from the connectors directory
Claude connectors directory	Admin portal; tools synced from the connected server	About 10 minutes, September 8	Community listing; 120 declared tools, no push or analytics	The synced tools did not match our description
mcp.so	Authenticated form and API validation	Update completed September 8	Corrected transport and overview live	HTTP 200 carried a validation failure
MCP Market	Paid remote-server submission	Live the next morning, September 9	Official listing live; a GoodBarber Skills card also listed	Try Now destination changed
Smithery	Official-vendor verification checklist	Proofs passed September 8	Public data still says `verified: false`	Passed proofs did not flip the public flag
skills.sh	No submission; CLI install telemetry	First install not timed	44 skills listed	Installs are not workflow success
Docker MCP Catalog	Remote-server PR, Docker team review, test credentials by form	Open since September 8	PR #4968 open; 0 comments	Review outcome unknown
VoltAgent skills list	Skills-list PR; rules require real community usage	September 8 to 14	PR #1031 closed without merge or comment	Adoption comes before the listing
BehiSecc skills list	Skills-list PR	Open since September 8	PR #689 open; 0 comments	Skills listings have their own review
Cline marketplace	Submission issue; review of adoption, credibility, maturity, security	Open since September 9	Issue #2492 open; 0 comments	Review weighs community adoption

The order we would follow next time

Start with the official registry, and with one description, one endpoint, one product page and one documentation URL, the same everywhere. Separate the hosted server from the skills package before choosing routes: some submission forms start with a GitHub repository, and a hosted server has nothing to put in that field.

Then read each gate before you submit. Replay public CI checks. If an agent prepares the PR, use the documented agent fast track. For a crawler, check the topic and the manifest. For a curated catalogue, read the acceptance criteria first, since some weigh community adoption, then keep the submission reference and wait for evidence of admission.

After publication, read what went live: the tool inventory, the authentication metadata, every destination link. Keep dates next to results and recheck weekly. The merged PR, the badge and the live page can tell three different stories.

All of it points to one page, the connection guide. If you are connecting GoodBarber to an assistant, start there.

One more thing: that was a dense week, and these procedures move fast. What you read above comes from each directory's own documentation where it has one, and from what we saw on the forms otherwise, checked on September 14. If you run one of these directories and something here is wrong, or already out of date, tell us in the comments.

Which check caught a mismatch in your own listing: the response body, the tool inventory, or the link after publication? Genuinely curious.

Fifteen years of the same click: what the agent era keeps rediscovering about distributed systems

Pierre- Laurent Medori — Wed, 16 Sep 2026 11:46:07 +0000

On September 11, I left a comment under Tilde Thurium's post about graph engineering. Tae Kim had described the moment the term clicked: the microservices analogy turned the graph from relationships in a knowledge graph into an agent's control flow, something you can inspect.

I replied:

Same click here, on several subjects lately. Fifteen years of systems architecture, and I still catch myself arriving, after a day of thinking, at something that was a reflex in microservices four years ago. But what strikes me most is how similar the subjects turn out to be.

The part I meant was the day of thinking. Recognizing an old problem eventually is not the achievement. Recognizing it before spending the day on it would be.

That week I had been discussing idempotent webhooks, then a reconciliation cron with a dead-letter queue, then graph engineering. Different conversations, different starting points, and each one ended on a mechanism I already knew, after I had spent time treating the problem as new.

Click one: the double charge

In a discussion on r/nocode about verifying integrations, the conversation came to duplicate effects when a webhook is retried. My first reaction: a second charge exposes a missing idempotency guarantee. Testing can reveal that gap. Sending another request does not fill it.

The follow-up was about concurrency. Fair point. Send the same event twice, one after the other, and the second request finds the record the first one left. That shows the check works after a completed write. The bug lives in the other case, two requests arriving together.

Both ask whether the event has been processed. Both see that it has not. Both proceed. Every line of code behaves exactly as written, and the system does the wrong thing twice.

For duplicate deliveries of one event, put a non-null database uniqueness constraint on its identity, scoped to the provider and the account where needed. Attempt that insert before the local business writes, and let only the successful insertion proceed. Commit the deduplication record and those writes in the same transaction, so a rollback leaves the event retryable. The database has to arbitrate the competing inserts; an application-level lookup followed by a write leaves a race. PostgreSQL's notes on uniqueness checks explain why the conflict check belongs inside the insertion.

The transaction boundary matters. Commit "processed" first, crash before doing the work, and the next delivery gets discarded forever. Move the record to the end without protecting the writes, and the duplicates come back.

An external charge or email crosses another boundary. Write the outgoing intent in the same database transaction with a transactional outbox; its worker can still deliver twice. Where the recipient supports it, retries need a stable operation key under that provider's idempotency contract, retention window included. A local unique row cannot enforce uniqueness inside somebody else's service.

For a projection of current provider state, I also prefer rereading the provider to blindly applying an old notification. Stripe does not guarantee event delivery order. Concurrent refreshes still need serialization, or a version check before writing locally.

The test I would ask for: deliver the same event concurrently, then inspect the actual effects. If the operation should create one record and send one email, count both. Then interrupt processing around the commit boundary and retry. Those are two different failure modes.

My bias in that conversation was out in the open: I said I come from the development world, and that I am probably biased toward solving this in the database.

I had arrived at a unique constraint again.

Click two: green all the way down

Under my piece on tracking state instead of an agent's history, Mudassir Khan described a webhook that confirmed delivery while the downstream consumer silently dropped messages on a schema mismatch. Clean delivery log, no data. The loss only showed up once a reconciliation step compared inbound and stored counts. The question that came with it was practical: can you wire reconciliation into an existing pipeline without a rewrite?

My answer was an independent second reader. A read-only job on its own schedule, reading the system of record and comparing what exists with what should exist. The pipeline keeps running; the reader starts exposing discrepancies.

A read-back inside the run shares the run's blind spots: same cache, same credentials, same definition of success. An independent reconciler derives the expected records from a durable source and reads the persisted results through a path chosen on purpose. A separate schedule is not enough on its own: the reader has to stay clear of the assumptions that produced the false success. What it produces is a discrepancy report, not one more claim that the run completed.

It is the reconciliation cron that microservices impose on you the day you give up distributed transactions. You need a way to discover that one side advanced and the other did not. The cron exists because nobody gets to infer agreement from their own successful request.

Counting is a good first check, and it has two holes. Matching totals can hide missing records offset by duplicates. Matching identities can hide objects with nothing inside. I keep one in a test app: a French draft with a title, a slug and zero paragraphs, left by a June translation run I remember as green. A count would have passed it; the draft exists. The reader needs a content invariant per object type on top of the count. In my case, at least one paragraph.

Upstream, a schema rejection should land in a dead-letter queue as a recoverable record, not in nothing. Once a fixed batch of unique messages has finished processing, one outcome per message, the accounting is simple:

inbound = stored + dead-lettered

While work is pending, that category belongs in the equation too. Count delivery attempts on one side and unique records on the other, and the equation means nothing. Dead-lettered messages also need an owner and a recovery path: storing the rejection does not repair the operation. The reconciler is the backstop, not the fix.

What struck me was how little of this answer depended on agents. I had been asked about a modern automation pipeline, and I had described a reconciliation job I would recognize in an older integration system.

Click three: the diagram with a new name

Then came the graph engineering discussion.

The useful part of Tae's comment was the shift in what the graph represents. Once I stopped thinking about relationships between pieces of knowledge, I could see the execution structure: which work can start, what it depends on, and what happens when a branch fails.

In the sense used in that discussion, graph engineering makes an agent workflow's control flow explicit and inspectable. Nodes are steps; edges and conditions describe the allowed transitions, including how parallel work rejoins. That structure gives the surrounding software places to enforce state contracts and failure handling, even when a node calls a model whose answer varies.

Tae's comment ended on the case I would test first: a fan-out where one node fails silently. Say a review workflow starts two checks and combines their results. One check returns; the other times out. What lets the join proceed? Does it expose the missing result, retry that branch, or mark the review incomplete? A diagram that leaves those decisions out has drawn the happy path, not the workflow.

I know those questions from service orchestration. Putting a model inside one of the boxes makes the box more interesting. It also gives me more reasons to care about what the arrows permit.

The model can help choose a next step, but that choice still needs a place in an execution contract. Otherwise the diagram describes what usually happens, and the actual control flow lives somewhere in a conversation.

The delay before recognition

Across those threads, the recurring problem was that each participant held only part of the evidence. A worker knew it had received an event. A sender knew its request had been acknowledged. A coordinator had a result from one branch. None of them held the outcome of the whole operation.

Agents add a particularly persuasive participant to that arrangement. It can explain why the task succeeded in a paragraph that reads better than the error message of the component that failed. I find it easy to give that explanation more weight than it has earned.

The analogy has a limit. A transaction can prevent a duplicate write; it cannot tell you the content deserved to be written. A graph can make a decision inspectable, not correct. The old mechanisms contain some of the failures. Judging the model is still extra work.

In Kaamelott, a French TV comedy set at King Arthur's court, Perceval falls back on "C'est pas faux", roughly "can't say that's wrong", whenever a word escapes him. I recognize something of myself there. Sometimes I understand the mechanism before I understand the new name. Sometimes the new name delays the recognition.

That is the part of my own comment I keep coming back to. Fifteen years did not spare me the day of thinking. They gave me somewhere useful to land afterward.

I would just like to hear the click a little earlier.

Which old mechanism did you last rediscover under a new name? Genuinely curious.

"Too many levels of symbolic links" with no symlink in sight: autofs, a bind mount and a cron container

Pierre- Laurent Medori — Tue, 15 Sep 2026 11:54:39 +0000

In September, our PHP crons started failing with an error that sounded specific:

Too many levels of symbolic links

That is ELOOP, the error a symlink loop gives you, so the first move was to look for a loop on /srv/apps/app-a/task.php. There was no symlink on that path. Nothing to untangle.

Paths and component names below are generic examples, not ours.

By the time we looked again, on September 10, the file was readable. The autofs service had just restarted, and the same file answered on /opt/code/app-a/task.php and on /srv/apps/app-a/task.php. That left a question the successful ls could not answer: what had to work for the second path to reach the first?

For context: I run engineering at GoodBarber, an app platform, and these crons run its background jobs. The short version of what follows: the code was already bind-mounted into the container, the paths the crons used still went through autofs, so we mounted the code directly at those paths and took them out of the automount map. If ELOOP sends you hunting for symlinks that are not there, look at your automounts, and at what the container's mount namespace can actually see.

The autofs layer inside Docker

The cron container's Compose configuration already mounted the application code from the host:

host /srv/code/app-a -> container /opt/code/app-a
host /srv/code/app-b -> container /opt/code/app-b

The application expects /srv/apps/app-a and /srv/apps/app-b. The image supplied those two paths through an autofs direct map:

# /etc/auto.master
/- /etc/auto.apps --ghost,--timeout=30

# Two entries in /etc/auto.apps
/srv/apps/app-a -fstype=bind :/opt/code/app-a
/srv/apps/app-b -fstype=bind :/opt/code/app-b

Any access to either path could trigger the automounter. The other entries in that map were NFS shares, managed by the same daemon.

Two layers of mounting for code that was already local. Opening a PHP file depended on the service that manages our network mounts.

Why an automount can return ELOOP

The error name is narrower than the mechanism behind it. In Linux v6.1, follow_automount() increments nd->total_link_count, the same counter symlink traversal spends, and returns -ELOOP once it reaches MAXSYMLINKS, which is 40. During a path lookup, every automount crossed counts against the symlink limit. No symlink required.

The autofs documentation describes the other half. When an autofs filesystem is visible in several places and the mounts created by its daemon do not propagate to the others, access from those other places "will likely result in the ELOOP error". The caller keeps meeting a trigger and never sees the mount it is waiting for.

That is how you get ELOOP without a symlink. Which of the two our incident took, we cannot say: the investigation tied the failure to an NFS incident and an unhealthy automounter, and we did not capture a kernel trace. What the configuration did show is the dependency itself, and it had no reason to exist, whatever upset the daemon first.

The fix stayed in configuration

On September 10 we added direct Docker binds at the paths the crons actually use. The relevant Compose fragment:

volumes:
  - type: bind
    source: /srv/code/app-a
    target: /srv/apps/app-a
  - type: bind
    source: /srv/code/app-b
    target: /srv/apps/app-b

We also gave the cron container its own copy of the automount map, with those two entries removed. It takes both changes: leave the keys in the map and autofs installs its triggers over the direct mounts again.

One detail of our image: the map path, shown here as /etc/auto.apps, is a symlink to the real map file. The Compose override mounts the cron-specific map over that target. The web containers keep the map they had.

The application teams maintain and deploy that code. This was an operations fix, so it stayed in the mount configuration and the application paths did not move. No PHP patch, no local checkout drifting away from what the application teams deploy.

Proving the dependency is gone

Roy from The IT Crowd has a suggestion: "Have you tried turning it off and on again?" We already had a readable file after a restart. The useful test was whether it still needed the automounter.

Inside the cron container, check each mount on its own:

findmnt -o TARGET,SOURCE,FSTYPE /srv/apps/app-a
findmnt -o TARGET,SOURCE,FSTYPE /srv/apps/app-b
grep -cE '^/srv/apps/app-(a|b)[[:space:]]' /etc/auto.apps

Both mounts should resolve to the host-backed code, with no autofs trigger underneath. The count should be 0, and grep exits with status 1 when nothing matches. If mounts are stacked, read /proc/self/mountinfo as well: the filesystem on top hides what sits below it.

Then check the cron runs after the deployment. The jobs have to actually run, and their new logs have to be free of ELOOP. Silence from a job that never started proves nothing.

The deployment was confirmed working on September 10. One check we have not run yet: in a maintenance window, with the NFS-dependent jobs paused, stop autofs inside the cron container, read the PHP entry point, then restart autofs and confirm it is active. The direct bind should stay readable. That tests the dependency removal, not an NFS hang.

The maintenance we kept

We now have two maps, and any change to their shared NFS entries has to land in both. Removing a runtime dependency left us a configuration chore.

The code was local all along. We changed how the crons reach it, and opening a local PHP file no longer needs the automounter to cooperate.

Have you met ELOOP with no symlink in sight? What was underneath on your side: autofs, a bind mount, something stranger? Genuinely curious.

How do you debug something that is allowed to be wrong?

Pierre- Laurent Medori — Fri, 11 Sep 2026 12:03:38 +0000

Yesterday afternoon a runtime I wrote created 70 paragraphs on the same draft article. It was supposed to create one. The runtime was an experiment in explicit state, the kind where the model gets the task spec, a small JSON, the last observation, and nothing else; it had a commit rule I was proud of: nothing counts as done until a read-back confirms it. After every paragraph it read the draft back, found no paragraph, and asked the model for another one. Seventy times.

My first suspect was the model. It was the last thing that had produced output, and blaming it costs nothing. Then I looked at the response sizes of the read-backs: 742 bytes, 742, 742, twenty-five times in a row, then 131,991 bytes at once. The paragraph list had been served from a cache with a 60-second lifetime. The model had done exactly what a correct runtime told it to do, and the runtime had been lied to by its own read path. Not the model. Not even the rule. The order of two reads.

One line of context so you know where I stand: I run engineering at GoodBarber, an app platform, and I operate a production MCP server plus a few scheduled agents of my own. In the write-safety piece I asked whether a 200 meant the right thing happened; in the determinism piece I argued that verification is the job now. This one is about what comes after the verification says no. A system that is allowed to be wrong cannot be debugged by asking whether it was wrong. It has to be debugged by three questions, in this order: what was it allowed to get wrong, which layer got it wrong, and how long did the error live.

Write down what it is allowed to get wrong

I had never written this down. Doing it took an evening and changed what I look at first.

block	output	tolerated	never allowed	detected today by	when it does not run
MCP server, as its operator	tool results with a policy envelope, one usage row per call	latency; a read served from the 60-second cache when no write preceded it; a client choosing the wrong tool; 429 above 1,000 calls a minute	a 2xx for a write that did not persist; a stale read right after a write by the same token; a tool silently missing from a client's list; a push nobody scheduled	Sentry on unhandled exceptions; status and error code in the usage row; one guard test against schema shapes that hide tools	loud: 421, 503, 429. Silent: the server is up and its cache is wrong
content agents, Claude Code sessions writing to seven blogs	drafts in `nonpret`, patches, JSON-LD	a clumsy sentence; a meta title to redo	a live article patched without a diff; a create that defaults to `published`; an em-dash in a slug; invalid JSON-LD on a live page; an empty body reported as done	my approval in the CMS; a diff before and after when a patch touches live pages; JSON-LD validation after injection	nothing ships. The risk is the half-written object
scheduled tasks: noon ops review, two code reviews, glossary sentinel	a DM to one colleague; two morning reports; a Monday report on 170 pages	a false positive dismissed in a minute; a missed run if the next one says so	"all green" over a broken page; a number in the DM that is not in Nagios; a skipped run with no trace	22 negative test cases and a positive control on the sentinel; the colleague reading the DM; nothing counts the runs	silence, indistinguishable from "nothing to report"

Two admissions fell out of the table. First, I have no fixed review routine: I read what an agent produced when it produced it, or when something surfaces, with no schedule and no written verdicts. Second, half of the "never allowed" cells had no detector the day I wrote them. Write it down, or every output is a bug and a feature at the same time.

Attribute before you debug

Here are the incidents I could date on my own systems since April, coded by the layer that was actually wrong. The list is what I could date, not a sample, and the split describes this list, not my fleet.

when	what	layer	first suspect
April 10 to 14	access tokens set to 300 seconds "just in case"; every client logged out every five minutes	configuration	the clients
April 8 to August 5	a per-app session cap counted sessions for an hour after their last call; clients that never sent the protocol DELETE starved it and legitimate traffic got 429s	harness	the clients, again
June	agents patched translations on seven live blogs with no draft step; a diff afterwards found 46 damaged paragraphs (from my notes; the artefacts are gone)	harness	the model
June	a translation pipeline reported green on a French draft with a title and no body; I remember the run as green, its history is purged, the draft is still empty	harness	the model
June 5 to 22	a schema change made for a directory review put a `oneOf` at the root of one tool; clients that drop such tools stopped seeing it for 17 days	tool contract	the clients
July 29 and 31	an unbounded Redis pool hit 10,000 clients, 3,650 connections rejected, every worker locked out; two days later one stream read per session filled the 200-slot pool	infrastructure	the load
August 5	a delete succeeded and the immediate read-back returned the object; fixed on August 10 by keying the read-after-write bypass on the token instead of a per-session variable	cache	the model, for a minute
August	the first version of my llms.txt annex read 8 days of logs and reported them as 121	the debugger	the logs
September 3	a state runtime created 70 paragraphs because its second read went through the cache	cache, then the runtime's read order	the model
June 3 to September 2	125 calls to 34 tools that do not exist, including `GBContent.getItems(sectionId, opts, onOk, onErr)`, from 5 apps	the model, with a harness that let it retry	nobody, until I counted

Ten lines. The model is the direct author of one of them, the invented tool names, and even there a client let it retry the same name until it gave up. My first suspect was the model in four of the ten. The reflex was wrong four times; the model, once.

The rule that fell out of it: attribute before you touch the prompt. Data and credentials first (was the input what you think it was), then tool and cache (did the read-back read the world or a copy of it), then harness and scheduler (did the loop do what the loop does, retry, truncate, reorder), then the model, then the human reading the output. Every layer above the model is deterministic and checkable in minutes. The prompt is where you go last, because it is the only layer where a fix cannot be verified.

The transcript is a witness, not evidence

When an agent has been wrong, the transcript is the first thing you open and the last thing you should trust. It records what the model saw and what it said. It does not record what happened. Ask the transcript "what did you call, with which arguments", never "did it work".

On September 3, 2026, I measured what our own read path answers, 50 cycles on a test app, every article created as a draft and deleted after. Create, read, delete, read: the immediate read after the delete still returned the object 3 times out of 50, after the August fix. Once in 50, the immediate read after a create did not find the new article at all. Then the sequence an agent actually produces, create, read, read, delete, list, read: the deleted object came back 50 times out of 50, and stayed for a median 61.1 seconds, maximum 61.2. The server bypasses the cache for exactly one read, the one that follows the write; a list in between resets it. A transcript of that agent would show a delete with a 200 and a read that returned the article. Both true. Neither what happened.

The same cycles confirmed three things I had logged in August as tickets. A delete returns only the policy envelope, no deleted, no id, no status; the transcript literally cannot show what was deleted. cms_create_article still defaults to published, so a create without an explicit status is a publication. And the verification hints on a create list cms_get_article, cms_get_article_paragraph and cms_get_event, which are the first three read tools of the family in alphabetical order, while the tool plan points its verification call at cms_list_cms_sections. Follow the plan to the letter and you verify an article by listing sections.

None of this is exotic, and that is the point. The June agents that damaged 46 paragraphs were not lying. They were reporting. Nobody had asked the state, only the transcript.

Measure how long the error lived

Error rates are the number everyone asks for and the least useful one I have. What changes behaviour is time to detection, per incident, per channel.

error	lived	caught by
stale read after delete, list in between	61 seconds, every time	a script, September 3
the annex that read 8 days as 121	hours	me, because a number looked too round
the 17-day invisible tool	17 days	a guard test written on June 22; who noticed first is not in the log
429s on legitimate traffic from the session cap	until August 5; the cap dates from April 9, the first victim is undated	logs, August 5
the empty French draft	since June, still there	nothing; I keep it
ops review not running on August 8, 9, 15, 16	26 days for the first one	me, counting transcripts on September 3
125 calls to tools that do not exist	92 days	a CSV export, September 3

Two rows have no detector at all, only a person who happened to count. The four missing ops reviews were two closed-laptop weekends; nothing said so, because a run that does not start writes nothing, and nothing is the same colour as green. The invented tool names errored 125 times in front of a server that logs every call, and the log was never read for that question. Time to detection is the metric that tells you which of your checks are actually running. We do not lower the error rate. We shorten how long an error lives.

What survives at three weeks is the other half of that measure. For my scheduled runs: the full transcript, locally, with token counts and tool calls, so I can answer "what did it do, in what order" for any day since August. For the server: one row per call with keys and sizes, never values, and no fingerprint of which agent made it. For the June pipeline: nothing; the execution history is purged, and my memory of a green run is the only witness. Three systems, three answers to the same forensic question, and none of them can answer "was it the right action". Only the read-back can.

The debugger is also non-deterministic

The uncomfortable part of debugging a system that is allowed to be wrong is that the thing you debug it with is allowed to be wrong too.

My llms.txt annex read the wrong window. My first E2 verifier divided every French body by an English length of one and failed six correct drafts. And the glossary sentinel, before it was a script, was a cloud routine created on August 12 that rewrote its own checker at every run: a verifier that is a slightly different program each week cannot be debugged, because there is nothing stable to debug. It ran twice, both green. I replaced it on August 17, on that design argument alone, with a frozen script, 22 negative test cases, a positive control, under version control. Four reports since, all green on 170 pages, 89 to 166 seconds each. The state is the sitemap. The validator is the test file.

Freeze the verifier first. It is the only component whose correctness you can actually prove, and the only one nobody wants to write. On one article, 20 identical translation calls at temperature 0 on September 3 gave one distinct output; the eight deterministic checks I run on a translation caught all six faults I injected, and one of them, an altered URL, was caught by exactly one check. A judge would have shrugged. The status code would have said 200.

What is not built

Not shipped on our server, still: a heartbeat on scheduled runs; a token fingerprint in the usage row, so that "which agent" has an answer; an outcome flag next to the status code, so that a 200 that did the wrong thing shows on a dashboard; idempotency keys, version-bound writes, a plan-then-commit object; the three tickets above, open since August 5, unchanged at our August 28 commit. What is built: the error budget table, the incident registry, the frozen sentinel, and a read path I now know to measure rather than trust.

When your agent was wrong last time, what did you look at first, the prompt or the data? Genuinely curious which way the reflex goes, and how often the model turned out not to be the culprit. I will answer with the registry and the read-back numbers, cycle by cycle.

Fourteen years of blog posts, seven languages, one laptop: an open-weight model did our hreflang backfill

Pierre- Laurent Medori — Thu, 10 Sep 2026 11:37:50 +0000

In May I had 5,892 blog posts spread over seven hosts, the oldest dated November 7, 2011, and not one field anywhere that said which post was the translation of which. Same company, same blog: French on fr.goodbarber.com, English on www, then es, it, pt, de and nl on their own subdomains. In our CMS each language blog is a separate site, and for fourteen years a translation was published as a new, unrelated article. Google's hreflang wants, for every article, the full list of its language versions, and it wants the list on every one of them. We had the articles. We did not have the list.

One line of context so you know where I stand: I run engineering at GoodBarber, a no-code app platform headquartered in Ajaccio, Corsica. I have no religion about where a model runs. This post is about one job where a two-year-old open-weight model on a laptop was the right tool, with the numbers to show it, and about why we are hosting a Hacktoberfest Fest on exactly this subject on October 21.

Why this is a judgment call, not a join

Every obvious key fails. The number at the end of each URL (-a1332 in French, -a1486 in English for the same article) is a per-blog counter. Publication dates are days apart for a translation, sometimes months. Titles are translated freely, and the monthly "What's new at GoodBarber" post carries the same title every month in every language. Slug overlap works when the translator kept the English words and fails precisely when they did their job.

A person would read the summary of the French article, read the summaries of the English candidates, and decide. That is a language model's job. The question was which one, and where.

The recipe: four scripts, one afternoon of GPU time

Hardware: a MacBook Pro with an M3 Max and 48 GB of memory. Runtime: Ollama. Two models, both open weights: gemma2:27b, Google's June 2024 release, 15 GB on disk at the default 4-bit quantization, under the Gemma terms; and bge-m3, BAAI's multilingual embedding model, MIT licensed, 1.2 GB, 1,024 dimensions. Around 600 lines of Python with httpx, no framework, no vector database.

1. Crawl, deterministically

One script dumps every article of every language blog into a JSONL file, one line per article, from the CMS listing API. It flushes each page as it lands and skips ids already on disk, so it survives being interrupted.

blog	articles	first post	last post
fr	1,085	2011-11-07	2026-04-24
en	1,083	2011-11-07	2026-04-24
es	893	2013-01-24	2026-04-23
it	892	2013-01-24	2026-04-23
pt	778	2013-01-24	2026-04-23
de	683	2013-11-25	2026-04-23
nl	478	2013-11-25	2026-04-23

5,892 articles, crawled on the morning of May 4, 2026.

2. Narrow before you ask

French is the pivot: it is the source language of most of our posts and the largest blog. For each French article and each of the six other languages, the candidates are the articles of that language published within 90 days of the French one, closest first, twenty at most. The model never sees the 1,083 English posts. It sees at most twenty summaries and picks one, or none.

DATE_WINDOW_DAYS = 90
TOP_K_CANDIDATES = 20
MIN_CONFIDENCE = 0.7

This is where most of the accuracy comes from, and it costs nothing. A retrieval step does not need to be clever; it needs to make the question small.

3. Ask one small question, in JSON, at temperature zero

The prompt, verbatim:

Match translations of blog articles between languages.

RULES:
- A match means the SAME article translated - same specific content, same arguments,
  same examples - not just the same broad topic or shared keywords.
- Two articles can share a keyword (e.g. 'no-code', 'GoodBarber', 'app') and still be
  DIFFERENT articles. Reject those.
- Compare the FULL summary, not just the title. Look for the same specific subject,
  same angle, same takeaways.
- If you are not sure it's the same exact article, set index=null.
- Return high confidence (>0.8) only when the summary clearly describes the same content.

SOURCE (fr):
title: ...
summary: ...
date: ...

CANDIDATES (en):
[0] title: ...
    summary: ...
    date: ...
[1] ...

Reply JSON only: {"index": <int>|null, "confidence": <0..1>}. Default to index=null when unsure.

The call:

r = await client.post(
    "http://localhost:11434/api/generate",
    json={"model": "gemma2:27b", "prompt": prompt, "format": "json",
          "stream": False, "options": {"temperature": 0.0}},
    timeout=600,
)
answer = json.loads(r.json()["response"])

Three rules around it. An answer under 0.7 confidence is discarded. An article can belong to one row only, so a matched URL leaves the candidate pool for everyone else. And after every pivot the CSV is rewritten atomically, with a sidecar file recording which (pivot, language) pairs were already attempted, so a Ctrl-C or a rerun never pays for a prompt twice.

That sidecar exists because I reran the thing several times while changing the prompt and the thresholds. With a hosted API, each rerun is a line on a bill and a rate limit to negotiate. Here it was a keystroke.

I measured the cost of one prompt again today, September 9, on the same laptop, with the same code and the same model: the French "GoodBarber vs Glide" article against its thirteen English candidates within the window, 1,342 prompt tokens, 16 tokens out. Cold, 24.7 seconds, of which 11.2 to load the model. Warm, 2.3 seconds. Both runs answered {"index": 2, "confidence": 0.95}, and index 2 is the right article. About 6,500 (pivot, language) pairs at that speed is roughly four hours of GPU time. The crawl ran on May 4; the CSV was last written on May 6 at 20:49. Sent to a hosted model, those eleven million or so prompt tokens would have cost somewhere between a couple of dollars and a couple of hundred depending on the model. Money was never the argument. The meter was.

4. Verify with a different model

The generator is the judge. It should not also be the reviewer. Three layers, cheapest first:

Deterministic checks. Host matches the column, slug has the expected shape, each URL appears once in its column, and the publication dates within a row span 90 days at most.
Slug overlap. A cell whose slug shares almost no words with the French slug, in a row where the other cells share plenty, gets flagged. It is a heuristic with known false positives on well-translated slugs; it only surfaces candidates.
A second model. bge-m3 embeds the title plus the first 300 characters of the summary for all 5,892 articles (an 82 MB cache on disk, once). For each row, pairwise cosine similarity between the cells. A cell whose mean similarity to its row-mates drops 0.15 below the others is an outlier; a row whose mean is under 0.55 is weak. In fix mode the script proposes, for each outlier, the article of that language closest to the centroid of the other cells, and only if it scores at least 0.70 against the centroid, at least 0.70 against the French pivot, and beats the current cell by at least 0.10.

Two models disagreeing is the review queue. A human reads the queue, not 5,325 cells.

What came out

	rows	en	es	it	pt	de	nl
French pivots	1,085
cells filled		959	766	760	688	616	451

5,325 of the 5,892 articles landed in a row, 90.4 percent. 347 rows are complete in seven languages, 151 more in six.

The gaps are older than the model. The Spanish, Italian and Portuguese blogs opened in January 2013 and the German and Dutch ones in November 2013, and the early years published a lot of local-only content that was never translated. So the 193 French posts of 2013 yielded 227 matched cells and no complete row, which is the right answer, not a miss. From 2019 on, more than half of the French posts have all six translations.

year of the French post	French posts	matched cells	complete rows (6 of 6)
2013	193	227	0
2015	142	542	0
2018	48	226	20
2021	64	354	51
2023	87	473	65
2025	26	155	25

Two implementations, one list

We had no ground truth. Nobody was going to grade 5,325 pairs by hand, and a validator written by the same person with the same assumptions grades itself. So over the same two weeks in May, a colleague, Marc, built a second implementation with nothing in common with mine: a different method, an embedding model and a nearest-neighbour index (pgvector) instead of a generator and a prompt, the same French pivot, a similarity threshold at 0.60, and a preference for the French article that shares the same thumbnail image, by hash, when one exists. No date window, no candidate list, no shared code, and no import of my CSV. Two lists, built blind. The rule was simple: when they said the same thing, we would stop.

I pulled the export of the second implementation today and diffed it against my May file.

Its export lists 1,137 rows today; 1,051 of my 1,085 French pivots are in it.
Of the 4,122 cells both pipelines filled, 4,037 are identical: 97.9 percent.
85 differ. I read the first 40 by hand: in 38 the second implementation is right and mine is wrong, the other two are a toss-up. The errors cluster in three families: serial posts (the monthly "What's new", the engine revision announcements, where a 240-character summary does not carry the edition), topic pairs published in the same window (two reseller posts, two native-ads posts, which my 90-day filter served up together), and a handful where the summary simply was too short.
284 cells the second implementation filled that mine had left empty, and 110 that mine filled and it leaves empty.

Two implementations that share nothing but the corpus, landing on the same answer 98 times out of 100: that was the stop criterion, and we stopped. Since June our blog sitemaps carry the alternates, rebuilt from that export on every regeneration, best-effort: if the export is unavailable, the sitemap is generated without alternates rather than not at all. As of today, on our seven hosts:

host	blog URLs in sitemaps	with hreflang alternates
www (en)	1,107	983
fr	1,084	1,022
es	945	844
it	943	854
pt	888	801
de	768	701
nl	561	536

5,741 of 6,296 URLs, 91.2 percent. The alternates live in the sitemaps only, not in the pages' <head>, which Google accepts as one of the three supported ways to declare them. One entry, as served today, shortened:

<url>
  <loc>https://www.goodbarber.com/blog/design-trends-2026-...-a1608/</loc>
  <xhtml:link rel="alternate" hreflang="fr" href="https://fr.goodbarber.com/blog/tendances-design-2026-...-a1439/"/>
  <xhtml:link rel="alternate" hreflang="en" href="https://www.goodbarber.com/blog/design-trends-2026-...-a1608/"/>
  <xhtml:link rel="alternate" hreflang="es" href="https://es.goodbarber.com/blog/tendencias-de-diseno-2026-...-a1137/"/>
  <xhtml:link rel="alternate" hreflang="it" href="https://it.goodbarber.com/blog/tendenze-design-2026-...-a1102/"/>
  <xhtml:link rel="alternate" hreflang="pt" href="https://pt.goodbarber.com/blog/tendencias-de-design-2026-...-a1341/"/>
  <xhtml:link rel="alternate" hreflang="de" href="https://de.goodbarber.com/blog/designtrends-2026-...-a1441/"/>
  <xhtml:link rel="alternate" hreflang="nl" href="https://nl.goodbarber.com/blog/designtrends-2026-...-a1435/"/>
</url>

The same seven lines appear under the French, Spanish, Italian, Portuguese, German and Dutch URLs of that article in their own sitemaps, which is what makes the set reciprocal.

What I would tell you about open weights, after this

Task design beats model size. The 90-day window and the twenty-candidate cap did more for accuracy than any model choice would have. A June 2024 model at 4-bit was enough because it was never asked to search; it was asked to compare twenty summaries.

Repeatability is a feature. Temperature zero and JSON mode gave me the same answer on the same prompt today as in May. Debugging a matcher that answers differently on each run is not debugging.

The meter changes what you build. The attempts sidecar, the reruns, the fix loop: none of it would exist at a price per prompt. Zero marginal cost is not about saving money on the final run. It is about how many times you are willing to be wrong on the way there.

Verify with a second, cheaper model. bge-m3 is 1.2 GB and MIT licensed. It never decides; it reviews. Two models with different failure modes are worth more than one bigger model.

Build it twice. When there is no ground truth and no budget to make one, two implementations that share nothing, not the model, not the method, not the code, not the author, are a test suite you can afford. Where they agree, you are done. Where they disagree, you have a review queue, and 85 cells is a queue a person can read. Gemma 4 was already on the same disk in May, by the way; I ran the two-year-old one because it felt faster.

Privacy was not the argument here. Blog posts are public. But the crawl hit an internal API on a 192.168 address and nothing left the LAN, so if your corpus is not public, the same recipe runs unchanged.

Hacktoberfest 2026 comes to Ajaccio, and we are hosting

Hacktoberfest changed shape this year. It is now run by Major League Hacking and DEV, in partnership with DigitalOcean, and it stopped counting pull requests. The 2026 edition is 300-plus in-person Fests plus a global online event, all about building with open source AI: write your first skills.md, build an open-source agent, fine-tune an open-weight model. The tagline is "AI belongs to everyone", and after the afternoon described above I have no argument with it.

GoodBarber is hosting the Ajaccio Fest:

When: Wednesday, October 21, 2026, 18:00 to 21:00.
Where: CampusPlex, 95 cours Napoléon, Ajaccio, in the middle of town.
Program: doors at 18:00; at 18:30 a talk on building with open source and open-weight AI, why it matters and concrete ways to start; Q&A and open discussion at 19:15; drinks and time to talk to the people around you from 20:00; end at 21:00.
Language: the talk is in French; slides and shared resources are in English so you can keep going with the wider Hacktoberfest community afterwards.
Price: free. Registration on the MLH event page is recommended. Open to working professionals and university students.

No conference format, no pitch. We are one of the few tech companies headquartered on the island, and most developers here work alone; the point of the evening is to get them in one room on a weekday. If you have contributed to open source for years, come. If you have only been curious about where to start, come with a laptop, install Ollama and pull one model beforehand, and the recipe above fits in an evening.

Fourteen years of posts got their hreflang from a 15 GB file and an afternoon. The interesting part was never the model. It was that nothing stood between me and trying.

Your agent's memory is a liability: track state, not history

Pierre- Laurent Medori — Thu, 03 Sep 2026 13:04:57 +0000

There is a French draft in one of my test apps that I keep like a fossil. Article 96924661: a title, a slug, zero paragraphs. A June translation run created it; I remember that run as green. I cannot check it: the n8n execution history from June is purged. What survives is a screenshot and the likeliest path, read off the canvas and never reproduced: the article is created before its paragraphs, so if the split step yields nothing, "Create FR paragraph" runs without error on nothing and the report node declares success. Green all the way down. The draft is still empty.

As I wrote in June, in the piece on automating an app with n8n and MCP, "our runs burned 40,000 to 77,000 input tokens per minute, past the rate limits of an entry-tier API account, whatever the model." The survivor was not an agent: deterministic plumbing, plus one model call per article. One line of context so you know where I stand: I run engineering at GoodBarber, an app platform, and we run a production MCP server everything below runs on systems I operate myself.

Then a preprint crossed my feed at the end of August with a headline number: 16 times fewer tokens, 94 percent once you do the subtraction, if your agent keeps state instead of history. I had lived the mechanism before the paper, so I read it back. Here is the cell the number comes from, my June job replayed five ways, what our server learned the hard way, and the rule the paper does not have.

The number is real. The cell matters more.

The preprint is SKILL.state, arXiv 2608.26263, v1 posted on August 26, 2026, by Sanket Badhe and Priyanka Tiwari at Google LLC with Jonghyun Chung at Purdue. At every step the model receives an immutable skill specification, the current structured state as JSON, and the last observation; the reasoning trace is discarded once the runtime has validated its state patch. The prompt stays roughly constant, so cumulative cost is O(T) instead of O(T²).

The 94 percent is exact and it is one cell. Table 1, the authors' synthetic Warehouse environment, Gemini-3-Flash, horizon T = 100: 65,408 cumulative tokens against 1,062,387 for the stateful baseline, a 16.2x gap. Same column at T = 10: 43 percent. On the public benchmarks, InterCode CTF and τ-Bench, the reduction lands between 11 and 66 percent depending on the baseline. The gain is a horizon effect.

The result that matters is elsewhere. At equal budget, roughly 1,800 prompt tokens for every runtime at T = 100, accuracy was 0.94 for the structured state, 0.52 for a capped summary, 0.22 for ReAct plus LLMLingua, 0.18 for a truncated window. The saving is a side effect. The structure is the result.

The prefix is the bill

On September 3, 2026, I counted our tool inventories with the count_tokens endpoint against claude-haiku-4-5. My n8n test app, 62 tools: 17,763 tokens. The larger test app, 77 tools: 23,174. The seven tools a translation needs: 2,416. One default page of cms_list_articles on the small app: 16,987. The paper's mean prompt per step, all three inputs included: 1,905.

A ReAct loop on that app opens every turn with nine times the paper's whole prompt in schemas alone, and each list it calls adds nine more that history never forgets. D's six-field state object counted 56 tokens; a create-and-read-back envelope, 112.

I replayed June in five arms

Same job, same six English articles, same test app, September 3, 2026, claude-haiku-4-5, prepaid credits; the table is run 1, except D, the best of its six versions. Every French draft was created as status: draft with a marker, then deleted. Verification is deterministic and outside the model: the five checks in the code block further down.

arm	what	verified	model calls	input tokens	cache read	list price
A	ReAct, full history, 62 tools	6/6	25	1,159,084	0	$1.267
B	ReAct, full history, 7 tools	6/6	25	893,341	0	$1.002
C	script plumbing, one call per article	6/6	6	7,135	0	$0.054
D	explicit state, best of six runtime versions	4/6	80	155,249	0	$0.348
E	arm A with prompt caching	6/6	25	123	1,080,099	$0.306

A does C's job with 160 times the input tokens. B, with 55 fewer tools, still needs 125 times: trimming the inventory does not fix a loop that re-reads its own transcript. A peaked at 694,434 prompt tokens in a 60-second window and never saw a 429; the account is not where it was in June, so I will not quote a June limit. C is the June survivor, a stdlib script now that the n8n workflow no longer runs: O(1) per article, and its prompt was the paper's triplet before anyone named it.

A second pass the same day reproduced A within 0.1 percent (1,158,626 input tokens) and B 13 percent lower (773,767); C used the same 7,135 tokens and E came out at the same $0.306.

E, arm A with cache breakpoints: 123 uncached input tokens, 73,114 written to cache, 1,080,099 read from it, 93 percent of the prompt served from cache, a quarter of A's price. Caching makes the transcript cheap. It does not make it small, or a sufficient statistic of anything.

And D, my own SKILL.state, never finished. Six runtime versions in one afternoon, 80 to 100 steps each, prompt flat between 1,400 and 2,500 tokens per step, best result 4 of 6. Each version failed somewhere new. The first schema carried ids and statuses but not the translation, so the model re-read the source at every step, 33 reads for 6 articles, and the runtime filed one French id under two sources. With the translation in state, the model invented tool arguments, author_id, body, language, and the server refused 74 creates in a row. With arguments filtered against the tool schema, it re-read and re-created: 52 reads, 17 drafts for 6 sources. With the runtime owning the state machine and naming the next expected action, it created 70 paragraphs on the same drafts: after each write the runtime read the article first and the paragraph list second, so the list came from the 60-second GET cache, empty, for about 25 cycles, then 131,991 bytes at once. My commit rule was right. My read path was the cached one. With the read-back taken off the cache, 4 of 6, then a patch rejected 52 times for a string where an integer was expected. The paper reports that on small open-weight models 68 percent of failures are premature overwrites of the state; on claude-haiku-4-5 mine were the runtime's, and every one of them was a validation the runtime did not do yet. The state was constant. Sufficient is the hard part.

My own transcripts are not O(T) either

On September 3, 2026, I scanned my local Claude Code transcripts, counters only, no content: 264 files, 175 sessions with at least 30 assistant turns, 32,607 turns. Prompt per turn (input, cache creation and cache reads) climbs until a compaction, then drops: 37 drops of more than half, a median cost of 2.2 times a constant-size state, a median cache read share of 0.968. The longest session ran 2,444 turns and 1,071,111,255 prompt tokens, 6.89 times a constant state, 97 percent of it cache reads. That is Daniel Vaughan's point about Codex CLI: O(T²/K) by chunks, not O(T). I have never noticed a decision lost to compaction and will not claim one. I can only vouch for the bill.

The server did it to itself first

Our MCP server stopped keeping history on August 10, 2026. Since then the transport is stateless: no Mcp-Session-Id, no session cap. What the server keeps per agent, keyed by the sha256 of the access token: the last ten tool names for one hour, enough to push the read that follows a write past the cache, and since August 11 a sliding counter of 1,000 requests per minute that answers 429 with Retry-After. Ten names and a number.

Measured on September 3, 2026, 50 cycles on a test app. A get straight after a delete: 3 stale reads out of 50, the August fix holds, not perfectly. Delete, list, get: the deleted object came back in 50 of 50 cycles and stayed a median 61.1 seconds, maximum 61.2, the 60-second GET cache doing what it was told. The bypass covers only the read right after the write; a list resets it. Three limits I read in the code rather than measured: another token never gets the bypass, nothing invalidates on write, and on an upstream failure the server serves stale cache on purpose. Read-back through a cache lies in both directions, as I wrote in the write-safety piece. It still does.

State is what was verified, not what the model believes

The rule I want, and the one the paper does not have: an agent's state is not what the model reports. It is what the world confirmed. Every write tool on our server returns _mcp_policy.verification_required: true, except six fire-and-forget push tools. D's runtime turned that policy into a commit rule:

commit rule, enforced by the script, never by the model:
  1. the model proposes a patch: {"set": {...}, "delete": [...]}
  2. the runtime validates it: an unknown key, a wrong type or any touch
     of verified_fr, and the patch is rejected, state untouched
  3. an id enters verified_fr only after cms_get_article passes every check:
     exists, status draft, at least one paragraph, no residual delimiter,
     body length within 0.6 to 1.8 of the EN body
  4. a failed read-back commits nothing; the failure is the next observation
  a 200 on cms_create_article is not a commit. The read-back is.

On one article, 20 identical calls at temperature 0 on September 3 gave one distinct output; the eight offline checks I run on a translation, a different set from the five runtime checks above, caught all six faults I injected, and a status code would have caught none. Deterministic checks, not a 200.

Running that rule against our own server hit the same holes I had logged in August, and the September 3 cycles confirmed them. Ticket B: a delete returns only the policy envelope, no deleted, no id, no status. Ticket C: cms_create_article defaults to status: published, unchanged since May 13. Ticket D: a create on September 3 returned cms_get_article, cms_get_article_paragraph and cms_get_event as verification tools, the first three read tools in alphabetical order, while the tool plan points its verification call at cms_list_cms_sections. Follow the plan to the letter and you verify an article by listing sections; follow the envelope and you may end up reading an event.

The June draft reads differently now: an invalid state patch, created without verified, accepted by a run whose only witness is my memory. IDs, not vibes.

My scheduled tasks never had a memory

My scheduled Claude Code tasks had the triplet before I read the paper. Each starts from a fresh context: a daily ops review that reads Nagios and BuildBot, daily code reviews of two workspaces, a Monday glossary sentinel. From local transcript counters over August 1 to September 2, 2026: the ops review ran 30 times, median 25 turns, 49 uncached input tokens and 1,529,904 cache-read tokens per run, about 14 tool calls a run. The two code reviews, 23 and 29 runs, read 14,689,395 and 10,637,775 cache tokens per median run. The sentinel, 3 runs, 820,312.

A few dozen to a few hundred uncached tokens per run, and the cached prefix is the rest of the prompt: spec, then observation. Where is the state? In Nagios, in git, in the CMS: the system of record, never the agent. The paper's architecture, state pushed one layer down, no validator anywhere.

The sentinel is the closest thing I have to a validator. A cloud routine created on August 12 ran twice, both green; I replaced it on August 17 on a design argument, not an observed drift: a routine that rewrites its script each week is a moving specification. The replacement is a frozen script with 22 negative test cases, under version control; its four reports, August 17, 25, 31 and September 2, were all green on 170 pages in 89, 139, 116 and 166 seconds. The state is the sitemap. The validator is the test file.

The honest list

The paper's third limitation is the one I hit last month: when the task is the trajectory itself, throwing the history away destroys the work. My llms.txt forensic needed 121 days of logs and 1,321 requests to find 26 AI-labeled hits; the facts existed only in the sequence, and no crawler state schema would have held them. There, the log is not the memory. The log is the deliverable.

What the paper is not: synthetic-first, a strong horizon effect, a stateful baseline that pads the transcript with a state block rather than using LangGraph idiomatically, single-agent only. What our server has not shipped: any of it. No validator in front of any state file, no idempotency keys, no version-bound writes, no plan-then-commit object, no semantic span attributes, no token fingerprint column in the usage log. Tickets B, C and D were still open at our August 28 commit. D never finished.

What is cheap, and what I am doing next:

☐ Weigh your tools/list with count_tokens. Compare with 1,905.
☐ Log the four usage counters per turn.
☐ Schema and validator in front of the state file.
☐ Commit state after read-back; deterministic checks, not a status code.
☐ Freeze and version the spec.
☐ When the trajectory is the deliverable, keep a ledger.

What does your prompt look like at step 50, and who decided what goes in it? If you run a scheduled agent: where does its state live between runs, what validates it before the next run reads it, and how big is it in tokens? I am genuinely curious whether anyone has a schema in front of that file, or whether we are all editing JSON by hand and hoping. I will answer with token counts where I can.

The agents in production aren't mine. Here's what their server sees

Pierre- Laurent Medori — Thu, 03 Sep 2026 11:43:19 +0000

Somewhere between June 3 and September 2, 2026, an agent asked our production MCP server for a tool called GBContent.getItems(sectionId, opts, onOk, onErr). Parentheses, parameter names, callbacks, the whole JavaScript signature, sent as the name of a tool. The server has never had a tool by that name. It answered with an error, the way it answered the 124 other calls to 33 other tools that do not exist: get_posts, push_send, skills/cms, tools/list as a tool, and a file path, /tmp/gb_args.json. Five apps, 125 calls, 92 days, one hundred percent errors. Somewhere, a transcript said "let me try a different approach" and moved on.

That is not in any demo, and it is not in most of the "lessons from production" threads either, because the person writing the thread usually built the agent. I run the other side. One line of context so you know where I stand: I run engineering at GoodBarber, an app platform, and we operate a production MCP server that customers' agents call all day. My own agents run on a laptop, not in production, and the laptop closes on weekends. Everything below is counted from systems I operate myself, with the window and the date in the sentence. No customer names, no app ids, aggregates only; the volumes are rounded on purpose, the ratios are exact.

The fleet, dated

Here is what "agents in production" means from where I sit, as of September 2, 2026.

what	where	engine	model	cadence	who presses	runs
public MCP server	prod	our code, customers' agents	theirs, never ours	continuous	the customer	~100k calls, 100+ apps (Jun 3–Sep 2)
content agents	laptop → prod CMS	Claude Code	yes	on demand	me (CMS approval)	not counted
noon ops review	laptop	scheduled Claude Code	yes	daily 12:04	Nagios/BuildBot → DM	30 (Aug 1–Sep 2)
two code reviews	laptop	scheduled Claude Code	yes	every morning	git diff → me	23 & 29
glossary sentinel	laptop	frozen Python	report only	Mondays	170 pages → report	4 green (since Aug 17)
SEO checkpoints	laptop	one-shot tasks	yes	J+21/28/42	Search Console → report	4 run / 7 sched.
product AI	prod	RAG + back-office	yes	continuous	the customer	dozens of apps

Seven lines. Two are production, and the model in the loop there is the customer's or the product's, never mine; the five others run on a laptop. One line has no model in the check itself, and it used to: the glossary sentinel started life on August 12 as a cloud routine that rewrote its own checker every week, ran twice, both green, and was replaced on August 17 by a script with 22 negative test cases. Not because it drifted; because a checker that is a slightly different program each week cannot be debugged, and I stopped it before it had a chance to prove that. Four reports since, all green, 170 pages each, 89 to 166 seconds.

The line that matters most is the first one, because it is the one I see least.

What a server operator sees, and what it cannot

Every tool call on our MCP server writes one row: app, tool name, client IP, status, duration, request and response sizes, and the keys of the arguments, never their values. No session id, because since August 10 the transport is stateless and there is no session. No token fingerprint, no user agent, no protocol version. That table is the whole of my visibility into other people's agents. I see the requests. I never see the conversation.

From that table, June 3 to September 2, 2026, our two test apps excluded:

Close to a hundred thousand calls from over a hundred apps and close to a thousand client IPs. The median day grew tenfold, from a few hundred calls in June to a couple of thousand in the first days of September.
62.8 percent of calls are writes. The top tool is cms_create_article_paragraph, then cms_update_event. Agents are not browsing our customers' apps. They are filling them.
Concentration: the top five apps hold 55.1 percent of the calls, the median app made a few dozen calls in three months, and more than one app in five was used on a single day and never again.
Several thousand push calls. A push notification has no undo. Agents sent, or scheduled, thousands of them through us in 92 days, most of them through a single tool.
Error rate 2.9 percent, 30 timeouts in close to a hundred thousand calls, and the 125 calls to tools that do not exist.

Now the number I actually wanted. In the write-safety piece I argued that a write is not done until a read confirms it, and our server tells every agent so: each write returns _mcp_policy.verification_required: true with the tools to read back with. Do they? I grouped calls into sessions by app and client IP with a ten-minute gap (a heuristic, the only key the table allows), and asked how many writes were followed by a read of the same family within 120 seconds.

41.0 percent, the six fire-and-forget push tools left out of the denominator, median delay 18.9 seconds. For content, 41.5 percent. For the shop, 8.3 percent. On the stateless transport alone, August 12 to September 2: 42.2 percent.

So the honest sentence is: our server asks for a read-back on every write, and on the wire, three writes out of five never get one. The policy is advice. The clients decide. And I cannot tell which client, because the table does not know: that column, the sha256 of the token the server already computes for rate limiting, is the first thing I would add, and it is not shipped.

Silence has three shapes

My own scheduled agents run on my laptop and leave their transcripts there, so those I can count exactly. From August 1 to September 2, 2026, counters only:

The noon ops review ran on 29 of 33 days. The four missing days are August 8, 9, 15 and 16: two weekends, a closed laptop. Nothing announced the absence; a run that does not start writes nothing, and nothing is the same colour as green.
One code review, weekdays only, ran 23 of 23. The other, daily, 28 of 33: the same weekends, plus August 2.
The sentinel ran 3 of 3 Mondays.

Three shapes of silence, then. Absent: the run never started, and the only witness is a gap in a folder. Green and empty: in June, a translation pipeline of mine reported success on a French draft with a title and no body; I remember the run as green, its history is purged, and the article is still empty in the test app. Unread: a report nobody opens is indistinguishable from a report nobody sent. The fix for the first shape is boring and I do not have it yet: a heartbeat on every run, absence as the alert.

The debugger fails too. My llms.txt piece in August was built on a forensic read of 121 days of logs; the first version of its annex read 8 days and reported it as the whole window. The logs were fine. My analysis tool was wrong, and I caught it before publishing only because a number looked too round. In June, agents patched translations on seven live blogs, live, no draft; a diff afterwards found 46 damaged paragraphs. The agents were not lying. They were reporting. Nobody had asked the right question.

The bill nobody itemizes

Per run, from the transcripts, medians over August:

agent	turns	tool calls	uncached input tokens	cache-read tokens	wall time
noon ops review	25	14	49	1,529,904	about 3 minutes
code review, workspace	122	71	336	14,689,395	about 23 minutes
code review, AI workspace	102	56	240	10,637,775	about 21 minutes
glossary sentinel task (runs the script, comments the report)	13	7	26	820,312	about 8 minutes

Read those columns together. A scheduled agent costs a few dozen uncached tokens a run; the rest of its prompt, one to fifteen million tokens, is the cached prefix re-read at every turn, and the code reviews re-read it a hundred times. The tokens are cheap at cache prices and the runs are on a subscription, so the bill I feel is not that one. It is the other column: the reports are read by a human. On a good week that is five minutes a day, six days a week, about half an hour, and most of it is spent confirming that nothing happened. On a bad week I have never counted.

The line item no calculator shows is maintenance, and the server's git log itemizes it better than I could. Since going live on April 10, 2026: a session cap that starved legitimate traffic with 429s until August 5, then became moot on August 10 when the transport went stateless; a rate limiter promised for "v1.1" on April 9 that arrived on August 11; two Redis incidents on July 29 and 31; a schema change made for a directory review on June 5 that silently hid one tool from some clients for 17 days, because those clients drop any tool whose schema has a oneOf at the root; an access token lifetime set to 300 seconds "just in case" on April 10 that logged every client out every five minutes until April 14. None of it is a model being wrong. All of it is the agent surface being a production system.

Who holds the button

Write down where the human is, or the demo will put one wherever it looks good.

Content agents: every draft lands as nonpret on our CMS and stays there until I approve it. Nothing they write reaches a reader without a human click.
The ops review and the code reviews: read-only by construction. They post to a person, never to a channel, never to a system.
The server: no mandatory approval flow. The customer's client decides how much autonomy the agent gets; the server advises (verification_required, a tool plan, max_retries: 1) and blocks nothing except what the credential cannot reach. A strict mode that refuses a write unless a read came first exists and is off by default.
Push: the six push tools return no verification hint, because there is nothing to read back. My own standing rule is send: "at", never now. Thousands of pushes went through in 92 days and I have no idea how many were scheduled versus immediate. That is a column I could add, and have not.

The honest list

No heartbeat on my scheduled runs, and a laptop is not a scheduler. Absent looks like green.
No token fingerprint in the usage table, so read-back coverage is a heuristic and per-client numbers do not exist.
The usage row is inserted on the hot path, before the response goes out, and is best effort: a database hiccup loses the row silently. No alerting sits on that table. No traces, no spans.
The server's read-back bypasses the cache only for the read that immediately follows a write, by the same token. On September 3, in 50 cycles on a test app, a get placed after a delete and a list still returned the deleted object 50 times out of 50, for 61 seconds. Three immediate read-backs out of 50 were stale too.
Not shipped, still: idempotency keys, version-bound writes, a plan-then-commit object, semantic span attributes. Prescribed in August, unchanged at our August 28 commit.
Ticket B, C, D from my August audit, all still open: a delete returns only the policy envelope; cms_create_article defaults to published; the verification hints on a create list cms_get_event because they are the first three read tools in alphabetical order.
34 tool names that do not exist were called 125 times. I did not know until I counted.

None of this shows in a demo because a demo is one run, watched, on a happy path. Production is the runs nobody watched, and the table that says what they did.

How do you know your scheduled agent ran this morning? Not that it was right: that it ran. Genuinely curious. And if you operate a tool surface for other people's agents: what share of their writes gets read back, and do you know which client is which? I will trade numbers, counting scripts included.

There is no free lunch, especially in app publishing

Pierre- Laurent Medori — Tue, 01 Sep 2026 11:54:27 +0000

2009, mobile networks class. We were studying how TCP handles packets lost over 3G: the network detects the loss, replays the packet, and the transfer survives a radio link that drops things all the time. Someone asked the obvious question: why not just make the radio link reliable in the first place? The professor's answer became the soundtrack of that semester: there is no free lunch. Reliability over 3G is not free, it is paid for in retransmissions, latency and battery. The cost never disappears. It just moves to wherever you are not looking.

I now run engineering at a no-code app builder. Every pricing decision I have seen from the inside since, his sentence explains better than any business book.

Here is the version of it nobody puts on a landing page: you can build an app for free anywhere, and you can ship one for free nowhere.

The asterisk every vendor knows by heart

Search "free app builder". Every result promises an app for free. Every vendor behind those pages knows exactly where their free tier ends: right before the App Store and Google Play. Build for free, preview for free, then the store gates close and the pricing table appears.

I will not do the pricing comparison here. We published a dated, sourced table of what each major free tier actually includes on the company blog: Can you really create an app for free?. Spoiler: the pattern is identical across the market, and one competitor even admits in its own comparison that store publishing always requires a paid plan.

This post is about the side of the counter you cannot see from a landing page: what the word "published" costs the vendor. Because that is where the free lunch dies.

What "published" costs us, forever

A built app is a file. A published app is a commitment. From our machine room, that commitment looks like this.

The store treadmill never stops. Google Play raises its target API requirement every year. Apple killed UIWebView in 2020. The legacy push protocol died in March 2021. The Play Data safety form arrived in July 2022, Apple privacy manifests in May 2024. Each event means rebuilding and resubmitting fleets of apps through two review pipelines. Skip one deadline and apps disappear from stores. This work is invisible, recurring, and nobody can bill it to "free".

Runtime is real hardware. Our customers' apps call a backend every day. We run our own machines in racks in European datacenters. The CDN moves terabytes per day. Push delivery counts in millions of notifications per week. An engineer is on call all year; the status page is public (goodbarberstatus.com). None of this pauses when a customer pays zero.

A free plan that included native publishing would mean carrying all of the above, indefinitely, for free. Nobody in this market does it. That is not stinginess. That is arithmetic.

So who pays for your free plan?

The lunch is never free; the bill just moves. On a typical free app builder tier, you pay with:

Branding. Your app advertises its builder. Removing the badge is a paid feature, everywhere.
The wall. The upgrade prompt lands at maximum investment, minimum leverage. That timing is not an accident, it is the business model.
A link instead of a listing. Free "apps" are often web links. The store fees never trigger, and neither does the store presence you came for.
Your time. The expensive one. Rebuilding elsewhere because platform one could not ship is the worst deal in no-code.

And two line items belong to nobody's free plan because they are not the builder's to give: Apple charges 99 $ a year for a developer account, Google Play 25 $ once. A page that promises store publishing "for free" without those numbers is describing a demo.

In defense of free tiers

Said plainly, because my employer's competitors deserve fairness: free tiers are a great deal when your goal matches their content. Testing an idea on a real phone screen. Learning how a no-code editor thinks. Shipping a link to five colleagues. For all of that, use a free plan and pay nobody, including us.

The problem is never the free tier. It is the landing page that lets you believe the lunch stays free all the way to the App Store.

The menu with prices on it

GoodBarber has no free plan, and now you know the reason is structural, not commercial. Our trade is different: the full platform, free for 30 days, no credit card, including a PWA you can actually publish during the trial (what free really includes). When it ends, nothing is charged, because we never took a card. You have seen the real product and the real prices, and you decide.

No free lunch. But a menu with prices on it, and a table where you eat for 30 days before ordering.

Same lesson as those 3G retransmissions: the cost never disappears, so honest design shows you where it sits. You know what to expect, and the price is not hidden. In 2009 we would have called that a well-designed protocol.

Determinism Is a Feature

Pierre- Laurent Medori — Wed, 26 Aug 2026 08:31:29 +0000

We spent seventy years building machines that do exactly the same thing every time. It took about three years of AI hype to start describing that property as a limitation.

I keep running into the framing in threads, in pitch decks, in hallway conversations: deterministic systems as the old world, rigid, unimaginative, waiting to be disrupted by something that improvises. And every time, I want to push back with the least fashionable opinion I hold: determinism is not the boring baseline that AI liberates us from. It is the single most valuable property a production system can have, and 2026 is the year we are most eagerly trading it away.

Let me be precise about what this post is not. It is not an anti-AI post. I run an engineering team, and we run agents in production: they draft, they call tools, they touch real systems every day. I like this work. The argument here is not "don't use models". The argument is that the more probabilistic components you add, the more load-bearing the deterministic parts around them become, and that almost nobody is hiring, promoting, or celebrating for the second half of that sentence.

What determinism actually buys you

Reproducibility is not an aesthetic preference. It is purchasing power. Walk through what "same input, same output" quietly funds:

A reproducible bug is a bug that is already half fixed. You can trap it, shrink it, watch it happen in a debugger. A non-reproducible bug is not a bug, it is a haunting. You don't fix hauntings, you appease them: add a retry, widen a timeout, sacrifice a sleep(500) and hope.

A failing test means something. In a deterministic system, red means broken. The moment flakiness enters, red means "run it again", and the entire signal collapses. A suite that people rerun until it passes is not a test suite, it is a slot machine with CI minutes.

git bisect works. Bisect is a miracle we stopped noticing: binary-search across history and the machine tells you which commit broke the world. The miracle rests entirely on replay giving the same answer twice. Bisect over a component that answers differently each run is not debugging, it is astrology with version control.

A diff tells the truth. Code review works because a change in behavior is legible from a change in text. The whole social contract of "I read your PR, therefore I know what it does" assumes the text determines the behavior.

Incidents can be reconstructed. The 3 a.m. question is always "what exactly happened, in what order?" Every answer you can give assumes the system will tell the same story twice.

Notice the shape of that list. Testing, review, bisect, rollback, audit: essentially every practice we call "software engineering" is a child of determinism. It is not one nice property among many. It is the property the rest of the discipline is built on.

The bottleneck moved

Here is the economic argument, because the craft argument never convinces anyone in a planning meeting.

For most of my career, producing the artifact was the expensive part. Writing the code, the config, the migration, the docs. So that is where the effort, the tooling, and the prestige went.

That cost just fell off a cliff. Code, tests, configs, entire services: generating a plausible version of almost anything is now nearly free. And when the cost of production approaches zero, the remaining cost of software is the cost of verification. Checking that the plausible thing is actually true is no longer a phase of the project. It is the project.

The asymmetry is brutal. Generated output scales with compute, which gets cheaper every quarter. Verification scales with human attention, which does not scale at all. Every team I talk to is discovering the same imbalance: the pipeline that produces changes got a jet engine, and the pipeline that validates them still runs on the same few pairs of eyes it had in 2019.

So the scarce skill inverted. The valuable engineer of 2020 was the one who could produce faster. The valuable engineer of 2026 is the one who can look at something plausible and determine, efficiently and reliably, whether it is correct. Plausible is the commodity now. Correct is still artisanal.

Put the non-determinism where wrong is cheap

None of this means banning models from production. It means placing them deliberately, the way you place any component with a known failure mode.

A probabilistic component belongs where a wrong answer is cheap and checkable: a draft, a suggestion, a summary, a first pass that a cheaper process can validate. It does not belong holding your state, moving your money, or standing alone in front of an irreversible action.

The pattern that works is deterministic rails around a probabilistic core. And the rails are built from the least glamorous toolbox in the profession: schemas validated at the boundary, state machines whose enums make illegal transitions unrepresentable, idempotency keys so a confused retry collapses into one effect instead of two, version-bound writes so decisions made on stale data get rejected instead of applied, a plan-then-commit gate in front of anything with blast radius, and an append-only log so "what did the system do?" never depends on anyone's memory.

I wrote a whole article on the how of those rails at one specific boundary, agents writing to production systems: where each mechanism goes, what each one catches, what the server contract looks like. Consider this piece the why. Nothing in that toolbox was invented this decade, and that is exactly the point: it is decades of transactional common sense, redeployed at a new boundary, the line between a model and everything you care about. The model gets to be creative precisely because the rails are not.

Read it back before you believe it

I have a rule I apply to everything now, and it started long before agents: a system's report about itself is a claim, not a fact. Read it back before you believe it.

The agent says it published the article? Fetch the article through the same path a reader would, and look at it. The migration says it ran? Count the rows. The dashboard is green? Green means "the things we chose to measure are within the thresholds we chose to set", which is a much smaller statement than "everything is fine". Your own code from last year? Same treatment. The author being you buys it nothing.

For years this was a habit. Recently I understood it is actually a design requirement, and this is where it connects back to determinism: verifiability is a property you build in, not a virtue you exercise afterwards.

A deterministic system is cheap to verify. One run is proof; the test you wrote once keeps testifying forever. A probabilistic system can be made verifiable too, but not for free: you have to design the read-back paths, the invariants, the reconciliation jobs, the alert that fires when the report and the reality disagree. Skip that work and you have not built a system, you have built a story the system tells you about itself.

The question that changed how I build is not "does it work?" It is "how will I know it is still working when I am not watching?" A system that cannot answer that question is not finished, no matter what the demo looked like.

Boring is a career strategy

Ten years ago, Dan McKinley told us to choose boring technology: spend your innovation tokens on your actual problem, not on your stack. The 2026 version of that advice is one level deeper: choose boring properties. Determinism. Idempotency. Reproducibility. Auditability. The stack will change under you; the properties transfer.

Because here is the career math nobody puts on a slide. When everyone on the team can generate code, generating code stops being what distinguishes anyone. The floor rose; the ceiling did not move. What is scarce in a room full of people who can produce plausible artifacts is the person who can say "this is correct, and here is how I know", and have the second half of that sentence be load-bearing.

That skill compounds, too. The prompt tricks of last spring are already quaint. The ability to design an invariant, bound a blast radius, or smell a non-reproducible test has been appreciating since the seventies and shows no sign of stopping.

Nobody demos idempotency keys. Demos are generation; production is verification. The distance between the two is where reputations are quietly being made right now.

Ship the boring thing

The systems I trust all have the same personality: they do the same thing every time, and they can prove it. When I add a model to one of them, and I do, I add it where being wrong is cheap, with rails around it that never improvise.

Determinism is a feature. Some years I would have said an underrated one. This year I will go further: it is the feature, the one that makes all the others checkable.

Ship it. And read it back before you believe it.

What is the most boring thing in your stack you would defend to the death? Genuinely curious.

The Cows Don't Send Alerts

Pierre- Laurent Medori — Thu, 20 Aug 2026 08:45:06 +0000

My neighbor spent yesterday morning clearing a patch of maquis with a bulldozer. I watched him from the fence for a while, coffee in hand, contributing nothing. Then I walked down to check that the cows had water, because it's August in Corsica and that is the one job on this farm that cannot wait until later.

The rest of the year I do engineering at GoodBarber, a mobile app platform. This month, my production system is a herd of cows and a water trough.

This is not a burnout story. I'm fine, the team is fine, nothing is on fire. This is about something I re-learn every summer at my parents' farm and manage to forget again by October: disconnecting is not the opposite of engineering. It's where my engineering comes from.

The job I closed the laptop on

For the past few months my work has looked like this: agents that draft, publish and verify things through MCP servers, pipelines that touch production, and one rule I repeat so often my team can probably recite it in their sleep: grep and eyes. When an agent says it did something, you go read the actual output, with your actual eyes, before you believe it. And a human stays in the loop for anything that ships.

I like this work. A lot. Which is exactly why closing the laptop for two weeks felt harder than it should have. When you love the loop, you stop noticing you're inside it.

The farm

The heat arrives before you finish your coffee. By nine the cicadas are at full volume and they won't stop until dark; after a day you stop hearing them, a wall of sound your brain files under silence.

The maquis smells like it's cooking. Immortelle, myrtle, hot dust. When the bulldozer bites into it the smell turns sharp, crushed green and diesel, and my father comments on the neighbor's technique from three fields away.

The trough is a twenty-minute walk, morning and evening. You put your hand in the water, because looking is not enough: the surface can shine while the float valve sits jammed. The cows watch you do this with monumental indifference.The soft tick of the electric fence. On the way back you kick the same pine cone the whole length of the path, because there is nothing else to do. It takes a few days to understand that nothing else to do is the entire point.

My phone lives in the kitchen. Half the property has no signal anyway. Evenings are long: tomatoes that actually taste of tomato, a card game nobody remembers the full rules of, chairs on the terrace facing the dark. When the cicadas finally stop, the silence has a texture. The silence in my apartment is just a machine that hasn't beeped yet.

Distance gives you back your judgment

From the hill behind the house you can see the whole farm at once: which field burned yellow, where the fence sags, where the water runs and where it doesn't. None of that is visible from inside the barn.

For months my days were a hundred items deep. A flaky deploy. A graph doing something odd. An agent behaving strangely at 2 a.m. Every one of those items legitimate, none of them wrong to look at, and together they formed a wall between me and the only question that actually matters: what shape is the system, and where is it sagging?

Two weeks of distance, and the shape comes back on its own. Without opening a laptop, I can tell you the two things that genuinely worry me for the autumn. In June I couldn't have, and in June I had all the dashboards.

Judgment turns out to be a renewable resource with a single supplier: distance from the thing you're judging.

Hard bugs die in boredom

Every engineer carries one around: the bug that survives every fix. You fix it, it comes back wearing a different stack trace. Mine came on vacation with me, uninvited. They always do.

The answer did not arrive at a desk. It arrived on day six, somewhere between the trough and the gate, wet hands, no phone. I didn't write a line of code. I just finally saw it: the whole mechanism at once, including the part every previous fix had been politely stepping around.

Boredom without notifications is not empty time. It's the only state in which the complete model of a hard problem fits in your head, because nothing is evicting it every ninety seconds. A notification doesn't need to be answered to cost you. It only needs to arrive.

We talk about deep work as if it were a scheduling technique. I'd call it something simpler: it's what your brain does by default when nothing is allowed to interrupt it. The farm doesn't teach you to focus. It just stops preventing you.

Urgent is a word that means something here

On a farm in August, urgent is unambiguous. Cows without water is urgent. The smell of smoke on the wind is urgent. Everything else waits, and waiting costs nothing.

The cows don't send alerts, by the way. You find out how they're doing by showing up, twice a day, whether anything is wrong or not. There is probably a monitoring philosophy hiding in that sentence, but I promised myself I wouldn't write it on vacation.

Before leaving I skimmed the threads marked urgent across my inboxes. Dozens. I already know what I'll find when I get back, because it's the same every year: most of them will have resolved themselves, gone stale, or turned out to be someone else's adrenaline. What remains is the real work, and it was there all along, hidden in plain sight under the pile.

Here's the uncomfortable part: I can't tell the two apart while I'm in the pile. Nobody can. Up close, all urgency produces the same cortisol. Telling signal from noise takes distance, and you cannot acquire distance while you're busy responding. The 90% of noise was never a triage failure. It was the absence of the one tool that makes triage possible.

Grep and eyes, with actual eyes

Here is the thing I actually came to say.

Checking the trough with your hand because the surface can lie to you: that is grep and eyes. Walking the fence line instead of assuming it held: that is verification before belief. The rule I repeat to my team all year was not invented at a whiteboard. It was installed here, a long time ago, by people who never wrote a line of code and would never trust a report about anything they could go and check themselves.

The farm and the job run on the same values. Look with your own eyes. Keep a human in the loop. Know the difference between urgent and loud. I don't disconnect to get away from engineering. I disconnect to visit the place my engineering came from, and I come back with the judgment, the unstuck bug, and a short list of the things that actually matter.

The neighbor finished his patch of maquis yesterday. Where there was scrub there is now clean ground, ready for whatever comes next. Two more weeks and it's my turn.

The first thing I'll do back at work is not open a dashboard. I'll go look at the system with my own eyes.