Lost updates in four MCP servers: GitHub, Grafana, PostHog and Kubernetes

Joshua Bryan — Fri, 13 Nov 2026 03:32:25 +0000

Two agents changed different fields of the same GitHub issue at the same moment, through GitHub's official MCP server. Both were told the write succeeded. In 19 of 30 pairs, one of the two changes was gone. GitHub's own per-field API, given the same two changes, lost none in 28.

I run IsoKron, which sells MCP audits. This is self-funded research, AI-assisted, and not commissioned or endorsed by GitHub, Grafana Labs, PostHog, the kubernetes-mcp-server maintainers or Red Hat. It was first published on isokron.ai.

We audited the MCP servers that GitHub, Grafana and PostHog publish for their own products, and the open-source containers/kubernetes-mcp-server. In the first three, two writes sent at the same moment to the same object could lose one of the changes, and both callers were told their write had succeeded. In the Kubernetes server, a full copy of an object re-applied through the tool without its resourceVersion, after another writer had changed a field, set that field back. This page puts the four results side by side. Each row links to the full audit.

The reason to look at this class of bug now is that it was rare when one person used one client. It stops being rare when several agents work on the same backlog, dashboard or project at once.

The four results

Server	What was tested	Lost through the MCP server	Through the platform's own API	Where the defect is
GitHub MCP server v2.1.0	Two updates to different issue fields of one issue	19 of 30 pairs	0 of 28 pairs, using GitHub's per-field update	In the MCP tool
Grafana MCP server v2.2.0	Two patches to different panels of one dashboard	30 of 30 pairs, on Grafana 13 and on Grafana 11	Legacy save endpoint: 30 of 30 on Grafana 13; 20 of 30 on Grafana 11, one save rejected in the other 10. Kubernetes-style API on Grafana 13: 0 of 30, one save rejected in every pair	In the MCP tool and in the legacy save endpoint
PostHog MCP server	Two updates to different fields of one insight	7 of 11 pairs, through the hosted server	6 of 11 pairs	In PostHog's API; the MCP server passes it through
Kubernetes MCP server v0.1.2	A full copy re-applied through the tool after another writer changed a field	Without `resourceVersion`, the other writer's change lost: replicas 10 of 10, a ConfigMap key 10 of 10. With the stale `resourceVersion` kept: refused with 409 in 10 of 10	The same body without force, straight to the API: 409 conflict, 3 of 3	Not filed as a bug: the tool applies with `force: true`, which upstream chose on purpose

Figure 1: Two simultaneous writes to one object, through three of the MCP servers and through each vendor’s own API

Group	Arm	Pairs that lost a change	Pairs with one call rejected (conflict)	Pairs
GitHub MCP server v2.1.0	Through the MCP server	19	0	30
GitHub MCP server v2.1.0	GitHub’s per-field update	0	0	28
Grafana MCP server v2.2.0, on Grafana 13	Through the MCP server	30	0	30
Grafana MCP server v2.2.0, on Grafana 13	Legacy save endpoint	30	0	30
Grafana MCP server v2.2.0, on Grafana 13	Kubernetes-style API	0	30	30
Grafana MCP server v2.2.0, on Grafana 11	Through the MCP server	30	0	30
Grafana MCP server v2.2.0, on Grafana 11	Legacy save endpoint	20	10	30
PostHog MCP server	Through the hosted MCP server	7	0	11
PostHog MCP server	PostHog’s API	6	0	11

Source: the table above, from the three linked audits.

A pair counts as lost when both calls returned success and one of the two changes was missing afterwards. All pairs were deliberately simultaneous. The figure covers the three servers tested with simultaneous pairs; the Kubernetes trials were a stale copy re-applied after another writer's change, so they are in the table only. None of these numbers says how often this happens in normal use.

Four different causes

GitHub: the MCP tool rewrites more than it was asked to change. To set one issue field, the tool reads every field value, merges in the change and writes the whole set back, with no version check. Two agents changing different fields overwrite each other. GitHub's own per-field update does not have the problem, which is why the control lost nothing. The MCP layer introduced the loss.

Grafana: the MCP tool switches off a check, and the endpoint behind it has a gap of its own. Patch mode reads the dashboard, applies the patch and saves the whole dashboard with overwrite forced on, so Grafana's version check never runs. Grafana rejected a stale save in 10 of 10 trials when the check was allowed to run. But two saves that arrive at exactly the same moment through the legacy save endpoint were both accepted even with the check on. Grafana 13's Kubernetes-style API did not have that gap: it rejected one of two simultaneous saves in 30 of 30 pairs.

PostHog: the loss is in the product's API. The same two requests sent straight to PostHog's API lost a write about as often as they did through the MCP server. The MCP server added nothing and removed nothing. An agent hits it more easily than a person does, because agents work in parallel.

Kubernetes: forced apply is upstream's choice, measured, not filed. resources_create_or_update applies with server-side apply and force: true. Upstream chose this on purpose (issue 799). The API has two protections for this edit: server-side apply without force refuses to take over fields another manager owns, and a resourceVersion in the body makes a stale write fail. Forcing switches off the first, so only the second remains, and only if the client keeps the resourceVersion it read. Neither native session in that audit ever sent a resourceVersion: in the concurrent-edit job, another writer changed MAX_CONNECTIONS from 100 to 200 about half a second after the agent read the ConfigMap, the agent's forced apply set it back to 100 four seconds later, and the agent told the user that the other keys were unchanged. Where the server does use optimistic concurrency, it held: resources_scale sends the resourceVersion it read, an injected conflict came back as an error with nothing changed (3 of 3), and real contention surfaced 10 of 10 times. A related design gap, K8S-04, is that resources_delete sends no uid or resourceVersion precondition, so it deleted an object recreated after the agent's read (10 of 10; kubectl has no uid precondition either).

The distinction matters when you read an audit of any MCP server. "The MCP server loses writes" can mean the server is wrong, that it faithfully exposes a backend that is, or that it made a deliberate trade-off. The control against the platform's own API is what tells them apart, and it should be in the report.

What happened next

Server	Reported	Status on 12 November 2026
GitHub	Issue 3751	Our fix is open as pull request 3781, with no maintainer review as of 12 November 2026. In our test of that approach a patched build lost 0 of 12 pairs
Grafana	Pull request 1518	Open; as of 12 November 2026 it was awaiting maintainer review, rebased onto the current main branch and mergeable. The proposed fix saves through the Kubernetes-style API where it exists and lost 0 of 60 pairs on Grafana 13. Grafana 11 has only the legacy endpoint; there the restored version check reports stale saves but still lost 14 of 30 simultaneous pairs
PostHog	Pull request 131519	Fixed for insights. A maintainer re-created the change as pull request 131620, with IsoKron as co-author, and merged it about 17 hours after we opened ours. Cohorts and dashboards lost writes in the same tests and, as of 12 November 2026, are not yet fixed
Kubernetes	Not filed	Forced apply is a deliberate upstream decision, and K8S-04 is a design gap; neither was filed. The research build adds `resources_patch` with an optional `resourceVersion` precondition, in the shape of upstream's open proposal 1352, and makes apply without force the default. That reverses a deliberate upstream decision, so it is a proposal, not a fix

PostHog's response is the one to copy: a small fix, reviewed and merged the next morning.

What this does not show

The counts are small: 11 to 30 pairs per cell for GitHub, Grafana and PostHog, with the two calls released together on purpose, and 10 trials per arm for Kubernetes (3 for the API control and for the injected scale conflict), on one machine, one cluster or one test account.
Real agents did task work through the server in all four audits. The lost-update counts did not come from those sessions: they come from scripted pairs and trials, so they show that a collision or a stale copy loses a write, not how often agents collide on their own.
Four servers is not a survey. We chose them because they are widely used and open source, not at random.
In the three simultaneous-pair cases the lost change can be put back by writing it again, and at Grafana it stays in the dashboard's version history. The harm is that nothing tells the agent or the user that it needs to.
The findings were found with AI assistance and checked as each audit describes. We sell audits of this kind, so read the method sections with that in mind.

How to check a server you run

Pick a tool that updates part of an object: one field, one panel, one tag.
Send two calls that change different parts of the same object at the same moment, from two separate server processes. Repeat at least ten times.
Read the object back. If both calls said success and one change is missing, you have a lost update.
Repeat with the same two changes sent straight to the product's API. If the API loses nothing, the MCP tool is the cause. If the API loses them too, the fix belongs in the API, and the tool should at least surface a conflict where the API offers one.

The full method, including the controls and the ways we got it wrong before getting it right, is in How to test an MCP server without fooling yourself.

Reproduce it

The method, with the controls and the mistakes we made first: How to test an MCP server without fooling yourself.
GitHub: isokron-ai/github-mcp-writepath-experiment is v2.1.0 plus the fixes as separate commits; go test ./... runs the upstream tests plus the ones added for each fix. Its concurrent-pair harness is tied to a private test repository and isn't published yet.
PostHog: isokron-ai/posthog-mcp-writepath-experiment, with the benchmark harness in harness/.
Kubernetes: isokron-ai/kubernetes-mcp-writepath-experiment, with the frozen harness; for b in baseline fixed research; do harness/build/build.sh $b; done builds all three binaries.
Grafana: the fix is pull request 1518 upstream.

If your numbers differ from ours, open an issue on the matching repository. Reproductions that disagree are the most useful feedback.

How this was tested. AI agents run much of the testing: scripted checks, attempts to disprove each finding, and real tasks that use the server the way an agent would. I also test every server myself on a separate machine and check each finding against the AI results. A finding appears here only if it holds up in both. — Joshua Bryan

For this piece: Every figure is taken from the four linked audits, which give the method, the controls and the limits for each.

Over to you

Have you seen a write tool report success and lose data? Which MCP server should we test next? Tell me in the comments.

DEV Community: IsoKron