Most teams think their DX is better than it is. Not because they're deluded but because they know too much. They know where the docs are, how auth works, what that error actually means. They've never experienced their own product as a stranger.
In this blog, I have describe the checklist I use to force that perspective. It takes about two hours. It will find things that embarrass you. That's the point.
When I audit a developer experience, the first thing I do is try to forget everything I know. I open an incognito window, grab a fresh API key, and pretend I'm a developer who just saw this product on a Reddit thread and has 20 minutes before their next standup. That constraint is everything (I like to call it stress-test because I am the one stressed).
Because that's actually how developers find you.
If you come from an engineering background and I do you already know what this feels like. You've been that developer. You've rage-closed a docs tab because the quickstart assumed you knew something you didn't. You've given up on an API not because the API was bad, but because getting to the first working call felt like too much work.
That experience is your most valuable tool as a DevEx engineer. Use it.
Before I run any checklist, I spend time thinking through the mental state of a developer hitting the product for the first time.
The zero-to-one moment
This is the thing I care most about. Everything else in the audit is important, but this is the one that decides whether a developer ever becomes a user at all.
✅ Can you find the docs from the homepage in two clicks?
I know it sounds trivial. It isn't. I've seen products with brilliant APIs where I couldn't find the documentation without using the search bar. If your developers have to search for your docs, you've already made them work harder than they should.
✅ Do you land on a quickstart, not a reference?
A reference is for people who already know what they're doing. A quickstart is for everyone else. If the first page a new developer lands on is a full API reference index, you've told them: "figure it out yourself." A quickstart says: "here's the most important thing, let's do it together." And that is why all the docs I write have a quickstart guide.
✅ Is auth explained once, clearly, in one place?
Auth is where developers get lost more than anywhere else. I've seen products where the API key setup is explained in the quickstart, the concepts section, and a help article all with slightly different instructions. Pick one place. Make it canonical. Link everything else there.
✅ What does the first error look like?
I do this deliberately and I send a bad request and read what comes back. Because this is what every developer sees the first time they make a mistake, which is guaranteed to happen. If the error is 400 Bad Request with no further detail, that's a UX failure. I write it down.
Docs quality
✅ Does every endpoint have a working example?
Not just the popular ones. Pick three at random from the tail of your reference docs. Do they have examples? Do the examples work?
✅ Are examples in at least two languages?
Python and JavaScript cover the majority of developers I've worked with. If your docs are curl-only, you're asking every developer to translate before they can try. That translation cost is real — not because it's hard, but because it's friction at the exact moment you want zero friction.
✅ Are error codes documented with actual fixes?
A table of error codes with one-line descriptions is not documentation. What I want to see and what I build when I have the ability to make the decision is error documentation that tells you why it happens, what the common causes are, and what to do about it. That's the difference between a developer fixing a problem in 30 seconds and opening a support ticket.
✅ Is the changelog maintained?
I check the date on the last entry. If it's more than 60 days old, I flag it not because the product hasn't changed, but because if it hasn't been documented, developers will be working with outdated assumptions. Trust erodes quietly when changelogs go stale.
✅ Do the docs match the actual API?
I run examples. Three of them, at random. Not the ones in the quickstart (those get tested all the time). The ones in the middle of the reference. If any of them fail, that's a critical finding. Docs that don't match reality aren't docs. They're misinformation.
SDK and tooling
✅ Does it install clean?
Fresh environment, one command, no flags. If I have to add --legacy-peer-deps or pin a version to get a clean install, I write it down. Because that's what every developer hits, and most of them won't know why (+ it shouldn't be on them to figure this out).
✅ Does the SDK version in the docs match the latest published version?
I check npm or PyPI. If there's a major version gap between what the docs show and what's published, every code example in the docs is _potentially _broken and developers won't know it until they hit a confusing error that has nothing to do with their code.
✅ *Is retry/rate limit handling documented or built in? *
Developers shouldn't have to implement exponential backoff from scratch. Either provide it in the SDK, or document the pattern explicitly. Both is better.
Error experience
I give this its own phase because I feel strongly about it. Error messages are UX. They're the moment where a developer is most frustrated, most likely to give up, and most in need of help. How you write your errors tells developers exactly how much you thought about them.
✅ Missing required field, what does the error say?
I remove a required parameter and send the request. Does the error say which field is missing? Or does it say invalid request? One of these is a 10-second fix. The other is a debugging session.
✅ Rate limit hit, what does the error say?
I hammer the endpoint until I get a 429. Does the response include retry-after? Does it explain what the limit is? Does it tell me how to request higher limits if I need them? Or does it just tell me I've been rate limited and leave me to figure out the rest?
✅ Do errors link to the relevant docs?
This is the one I push hardest for when I'm in a position to make the call. An error that links to its own documentation is worth ten well-written help articles, because it finds the developer at exactly the moment they need it.
Last but not the least: Time to first value
Time the whole thing.
I time it. From landing on the homepage to a working API call. I don't skip steps, I don't use internal knowledge, I don't ask anyone for help. Whatever number I get, that's the number.
✅ Count every gate.
Email verification, account approval, plan selection, sales contact requirement, waitlist. Every one of these is something I'd have to justify keeping if I had the authority to remove it. Some gates are necessary. Most aren't. Write them all down.
✅ Does a sandbox exist?
The cognitive cost of setting up a local environment is real. A browser-based sandbox removes it entirely. If I can try the API before I write a line of code, my likelihood of continuing goes up significantly.
✅ Is there a sample app to clone?
Developers copy-paste their way into new technologies. That's not laziness, it's efficiency. A well-maintained sample repository that runs in five minutes is the highest-leverage thing a DevEx team can ship. Give developers something good to copy.
✅ Is pricing visible without signing up?
I shouldn't have to create an account to understand what something costs. Hiding pricing creates a trust gap before I've even tried the product.
What I do with the findings
After this, you'll have a list of failures. Some will be obvious ("our error messages are useless"), some will be subtle ("the changelog is three months stale"), some will be structural ("auth is documented in four different places").
Prioritise by the metric that matters most: time to first value.
Everything that sits between a developer and their first working API call is a critical fix.
The audit is most useful when you do it with someone who hasn't worked on the product. Better still: watch an actual developer try your API for the first time, say nothing, and write down every moment they slow down or get confused. That's your entire roadmap.
Best,
Joe.
Confidence level: high. Hallucination probability: non-zero.
Top comments (0)