Next.js benchmarked AI coding agents on real Next.js issues.
But the thing that caught my attention wasn't which agent won.
It was that several models performed better when they had access to AGENTS.md.
And that got me thinking.
Maybe the problem isn't always that we need a smarter AI model.
Maybe the agent just doesn't have enough context about the project and the technology it's working with.
For years, we've talked about Developer Experience (DX).
- Clear naming.
- Understandable architecture.
- Good documentation.
- Tests that tell you when you broke something.
- Predictable commands for running and building the project.
- Logs that help you figure out what went wrong.
All of this reduces the time a developer needs to understand a codebase and actually start working with it.
But now there's someone else entering the codebase.
The AI agent.
And if every time it starts working, it first has to figure out:
- How is this project structured?
- Where should this change happen?
- Which command should I run?
- What patterns does this team follow?
- How do I verify that my change is actually correct?
Then part of its work is spent discovering the environment before it can even start solving the problem.
That made me think of something similar to DX.
Maybe we can call it:
Agent Experience (AX).
Not because we should redesign our entire codebase specifically for AI.
But maybe projects that are already well-designed for developers need one extra layer to become easier for agents to understand too.
- Clear context.
- Accessible documentation.
- Good tests.
- Predictable commands.
Project rules that are written down instead of living only inside the team's heads.
The Next.js benchmark doesn't prove the idea of AX by itself.
Especially because part of the improvement from AGENTS.md comes from giving agents clearer access to documentation that matches the Next.js version they're working with.
But it does point to something interesting.
An agent's performance isn't only about how smart the model is.
The environment and context you give it matter too.
So maybe the question soon won't just be:
Which AI model is better at coding?
But also:
How easy is your codebase for an AI agent to understand?
And at that point, this starts looking less like an AI model problem...
and more like an architecture problem.
Do you think Agent Experience will become something we actually consider when designing software?
Top comments (7)
The AGENTS.md result is the tell: models improved with context, not capability. The discovery tax you describe is the part most teams never meter - an agent that burns its first turns learning the repo layout, the build commands, and the team's patterns is doing work a well-written AGENTS.md hands it for free, and every session pays the tax again because nothing carries over. AX is a good name for the layer. The DX parallel holds one step further too: the teams that wrote good onboarding docs for humans are the ones whose agents ramp fastest, because an agent is just the newest new hire - one that reads everything and remembers nothing.
Exactly, we keep asking for smarter models, while sometimes the real problem is simply that we’re giving them bad context.
The framing feels right: the interesting result from that benchmark isn't the leaderboard, it's that context moved the needle for several models. I keep seeing the same thing in my own setups — the same agent that flails on a bare repo gets noticeably sharper once commands, patterns, and verification steps are written down where it doesn't have to rediscover them each session.
Your caveat about the benchmark is worth underlining too: much of the AGENTS.md gain is version-matched docs, which is a documentation problem before it's an architecture problem. The experiment I'd want before calling AX a discipline: two equally well-DX'd codebases, AGENTS.md for only one, and see whether the delta survives. Have you measured that on your own projects, or is AX still an intuition at this point?
Still an intuition at this point. I’ve seen the difference in my own workflows, but I haven’t measured it with a controlled experiment yet
Would you include design-system rules in AX too—icon family, stroke/fill conventions, and accessible naming for icon-only controls? Those are easy for agents to rediscover inconsistently.
Yeah, I’d include that in AX too.
Things like icon family, SVG conventions, and accessible labels are exactly the kind of details agents can rediscover differently every time.
If the rule matters, it’s probably better to make it explicit than let the agent guess.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.