We Audited 25 Global SaaS Platforms for llms.txt v2: Here is What We Found

GrowNexus — Sat, 19 Sep 2026 06:09:55 +0000

When the llms.txt proposal emerged as a standard to help LLMs and autonomous agents navigate web documentation efficiently, it promised an alternative to messy HTML scraping.

In August 2026, the specification was formally updated to v2, introducing two major changes:

Standardized discovery via HTTP headers and HTML <link rel="describedby"> tags.
Markdown twin conventions for individual documentation routes.

Our research team at GrowNexus audited 25 major SaaS platforms to measure how the industry is actually adopting the standard.

📊 Key Findings from the Dataset

We examined 25 prominent platforms across developer tooling, CMS, infrastructure, and productivity software:

Adoption Rate: 18 of 25 (72%) publish an llms.txt file.
File Size Variance: File sizes ranged from 2,088 bytes to 210,736 bytes (median: 16,299 bytes).
Spec Structure Compliance: Only 12 of 18 (66.7%) follow the section hierarchy (H1, blockquote summary, curated H2 list) outlined in the spec.
Header Discoverability (rel="describedby"): Only 2 of 18 (11.1%) emit the header.
HTML-Level Discoverability: 0 of 18 (0%) implemented <link rel="describedby"> in their page markup.

⚠️ The Problem with 100KB+ `llms.txt` Files

The most significant operational issue discovered is using llms.txt as a raw sitemap dump.

When autonomous agents (like Claude Code, Codex, or Hermes) parse context, scratchpads are token-budgeted. Files over 50KB consume 15,000+ tokens before the agent executes a single tool call, resulting in aggressive truncation.

A well-architected llms.txt should act as a curated index of core endpoints—not a full site mirror.

🛠️ Open-Source Benchmark Dataset

We have released the complete dataset and analysis scripts under the CC BY 4.0 license:

GitHub Repository: github.com/grownexus/llms-txt-benchmark
Complete Implementation Guide & Code: GrowNexus Insights

How is your engineering team approaching llms.txt implementation for your APIs?

DEV Community: GrowNexus

We Audited 25 Global SaaS Platforms for llms.txt v2: Here is What We Found

📊 Key Findings from the Dataset

⚠️ The Problem with 100KB+ llms.txt Files

🛠️ Open-Source Benchmark Dataset

⚠️ The Problem with 100KB+ `llms.txt` Files