DEV Community

Cover image for Agent Surface Auditor checks to ensure your site is ready for Agents Scans visit (must be)
Ntty
Ntty

Posted on

Agent Surface Auditor checks to ensure your site is ready for Agents Scans visit (must be)

Most discussions about "optimizing for agents" still sound too abstract.

In practice, agent-facing failures are usually plain engineering problems:

  • one machine-readable document links to one canonical path
  • another links to an older or duplicated public path
  • your sitemap forgets the important execution docs
  • your OpenAPI file is reachable, but buried behind inconsistent references
  • your best instructions are written for humans, not for machine execution

That is why we built a standalone Agent Surface Auditor.

It checks:

  • discovery files like robots.txt, sitemap.xml, llms.txt, and llms-full.txt
  • project-specific docs defined by config, such as execution docs or OpenAPI files
  • execution surfaces for MCP, API docs, OpenAPI, or CLI
  • content quality signals including heading structure, readability, and semantic HTML
  • canonical drift between repo state and public output

One thing we learned quickly: the interesting issues are rarely exotic.

They are usually:

  • stale public links
  • multiple canonical entrypoints
  • machine-readable docs missing from sitemap/index layers
  • content that is technically accessible but still too noisy for an agent to use efficiently

We also made provider choice explicit:

  • direct-fetch
  • browserbase
  • cloudflare
  • auto with quality-aware fallback

That matters because not every site needs browser rendering, and some providers succeed where others do not.

If you want the shortest possible install path:

`npx agent-surface-auditor run --mode url --url https://example.com --provider auto`
Enter fullscreen mode Exit fullscreen mode
`npx openskills install Citedy/agent-surface-auditor`
Enter fullscreen mode Exit fullscreen mode

The bigger point is this:

Agent-readiness is not just about publishing llms.txt.

It is about building a coherent execution surface:

  • one canonical path
  • one clean machine-readable layer
  • one obvious next action

That is a docs problem, an infra problem, and a product reliability problem all at once.

Fact Anchors

  • Agents benefit from a stable discovery layer and canonical machine-readable links.
  • Markdown and plain text reduce presentation noise for high-value technical content.
  • MCP, APIs, and CLI workflows are stronger action targets than generic landing pages.
  • Public skills should be installable through a direct command or registry flow, not hidden behind manual setup.
  • Cross-file link drift between discovery files, sitemap entries, and execution docs is a concrete failure mode.
  • Teams should audit agent-facing surfaces continuously, not treat them as one-off marketing assets.

Top comments (0)