A few weeks ago I pointed an AI at a vector database's documentation and told it to find everything wrong. It returned 47 findings in a couple of minutes- broken code, missing sections, explanations that would lose a beginner. As it often happens with AI, I was impressed for about as long as it took to start checking them.
About a third didn't survive a second look - the model (Opus) flagged a "contradiction" between two
sentences on one page — one was about search results being equivalent and the other about them being processed more efficiently. Both true, no conflict. It told me a parameter
was undocumented when there was a note explaining it three lines down. It wanted me to fix things that were already right.
The findings that were real — a deprecated API call, a quickstart shipping a literal
{collection_name} placeholder, an arithmetic error in a capacity formula- were genuinely
useful. However I only knew which oens to trust because I'd read every page myself. The value of spared time wasn't in the generation, but the verification.
And then i did think a lot about the nature of things for an hout or so - particularly about Ai and jobs - what we all think about occasionaly.
Is AI coming for the DevRel job? Yes. Is that the whole story? No (it usually never it). Which part is it coming for - thats the question.
Here's the honest version. What AI is good at now is the thing that used to fill my calendar.
It writes a competent getting-started tutorial. It drafts API reference. It explains what an
embedding is in a clean paragraph. For years "developer relations" quietly meant "content
mill" — ship the tutorial, ship the blog, ship the next one — and a model can do the first
draft of all of it.
I thought that would scare me - the AI presence in our everyday professional ives, but it actually brings relief. If the production part gets cheap, the role gets to be about the part that was always the actual point. I've been
trying to name that part, and I keep landing on three things. None of them is writing.
The first is judgement. The docs audit is the small version of it: plausible is not the same
as correct, and someone has to know the difference — which finding is real, which example will
quietly mislead a beginner, which "best practice" is two versions out of date. That comes from
having built the thing and sat with the ways it breaks. A model produces confidence at volume.
It can't tell you when its own confidence is wrong.
The second is reading the room. A model sees the text in a Discord thread. It doesn't feel the
thread. It can't tell that a respected person has gone quiet, that the same frustration is
surfacing in three channels in a tone curdling from "confused" to "done," that the loud
complaint is noise and the one-line aside is the real signal. I spent years inside blockchain
communities — some of the most opinionated, unfiltered audiences in
software, where feedback arrives fast and feelings are load-bearing. You learn to read mood. Which quiet is people thinking, and which quiet is people leaving. No model
does that yet.
The third is knowing which developer you're actually talking to. Ask an AI for a RAG tutorial
and you get the average one: Python, the common path, no opinion. But there's no average
developer. There's the ML engineer who wants it to work end to end on the first try, the data
scientist with ten million embeddings already sitting in a numpy array, the TypeScript
developer tired of every example assuming they'll spin up a Python server. Serving them means
taking a side about who you're writing for — and noticing when a whole group is being quietly
left out. AI averages. The job is to choose.
There's one more, and it's the one I care about most, even though it's the least glamorous:
treating docs and developer relations as a product you measure, not a feed you publish into and
forget. The numbers most content people never look at. Time to first successful query.
Self-service rate in the support channel. Which pages show up before someone sticks around, and
which show up before they leave. I built an ecosystem health dashboard once that tracked exactly
this kind of signal — activity, growth, where teams got stuck — and it moved strategy in a way
no count of published posts ever did. Run docs as a product and "we shipped 12 tutorials this
quarter" stops being a win. "People reach their first working query 40% faster" starts being
one. AI can write the tutorials. It can't tell you which number matters, or have the taste to
kill a popular page that isn't helping anyone.
Why write this down? I'm not totally sure. Partly to make myself say it clearly instead of
half-thinking it on the way to something else. Partly because I use these tools every day — I
used one for that docs audit — and I'd rather be honest about what they did and didn't do than
pretend in either direction.
Top comments (0)