AI in docs: Documentation dynamic shift most people seem to ignore

I’ve worked as a senior technical writer for over 6 years. Believe me when I say this: The consumers of documentation have shifted from users/developers to AI agents.

Two years ago, the question on everyone’s mind was “Will AI replace technical writers?” Now it’s not a matter of ‘will’ but ‘when’. We’ve seen a lot of this happening in the tech ecosystem lately. Companies are laying off their entire technical writing team in place of AI-driven workflows.

While there’s an entire conversation to be had here, it’s a topic for another day.

Today, let’s talk about documentation readers. Who actually consumes doc content?

Who are the primary users of documentation?

Who consumes documentation content?

When structuring information architecture for a documentation project, particularly a devtool, we often account for the user journey.

In recent times, however, that has not been the case. The average developer would query their AI coding agent for details on how to accomplish a specific task. Even when they’ve discovered your product, they’ll most likely want to use an easier way to find resources on your site, rather than searching through docs to find the request format for a specific task.

This shift means the primary consumers for docs are slowly becoming AI models.

What this means for documentation teams

You might be wondering… How does this impact me as a technical writer?

Let me be clear, it does affect how we work as technical writers. Companies that’ll stay ahead in this AI age are those that optimize for AI agents. You need to make sure your docs are easily discoverable to AI models.

The way AI models recommend content is somewhat different from traditional search engine optimization.

For instance, people usually ask AI agents direct questions on how to solve a specific task. So they’ll recommend brands that are direct in their content.

How can you optimize your content for AI discoverability?

There are different ways to make your content accessible to AI agents. The most common approach is to add an llms.txt directive to your docs site. When users or AI agents navigate to yourdocs.com/llms.txt, it should contain details about your site structure and links to relevant pages.

This works similarly to having a sitemap.xml file for search engines.
Aside from adding an llms.txt file, these are additional content optimization techniques.

• Serve your docs in markdown format: AI models often prefer reading documentation in .md format because it saves tokens. However, web content is being served as HTML pages. To stay ahead, have an .md version of your docs and include it as a directive in your site’s llms.txt file.
• Content negotiation: When you have a .md version of your site, make sure the content matches the web version. If agents discover that the content differs, it might send the wrong signal.
• Content structure: In AI SEO, content structure plays an important role. Your content should be direct and avoid unnecessary fluff. FAQs are an untapped gold mine. Incorporate FAQs into all documentation pages that directly address common questions related to the doc.
• Build AI tools: This is optional but recommended. Tools such as an MCP server equipped with your documentation would greatly help users and AI agents find your documentation and product details easily.

What tools can I use to check my documentation site for AI friendliness?

Dachary Carey, along with community contributors, maintains an agent-friendly documentation specification. This spec includes instructions on common patterns AI coding agents use to find docs and answer user queries.

The AFDocs tool integrates these specifications and provides a testing environment to check your docs against the specification.

Here is the command to run the docs Agent-Friendly test against the documentation spec:

npx afdocs check https://docs.example.com --format scorecard

When running the test, replace docs.example.com with the documentation site you intend to test. The tool provides a score for your doc, along with instructions and suggestions to improve it.

In conclusion

Documentation teams should begin to factor AI agents as the primary readers of content on their site, particularly developer documentation.

When planning information architecture and content structure, keep these recommendations in mind.

Are there other tips for building and maintaining AI-friendly docs you’re using? Please share with us in the comment section. I’d love to see how teams are navigating the advent of LLMs.