DEV Community: Yurii Serhiichuk

Your llms.txt guide is out of date: what v2 changed

Yurii Serhiichuk — Mon, 24 Aug 2026 12:28:52 +0000

TL;DR: llms.txt v2 landed on 10 August 2026, and two of the things the guides taught you about the format are no longer true.

The spec moved and nobody said anything

So you added an llms.txt at some point, most likely from a guide, and it's been sitting at the root of your site ever since doing whatever it is that it does. llmstxt.org now reads "The /llms.txt file, v2" and carries a modified date of 10 August 2026. The whole v1 to v2 diff lives on a separate changes page, which the spec links exactly once, in a single sentence in the Background, and that's the only pointer to it you're going to get.

Which means every guide written before that date describes v1 as current, because it was. That's not a criticism of the guides. The problem is the shape of this particular topic: you write your llms.txt once, the article that told you how to write it keeps ranking, and nothing anywhere tells a reader that the ground moved under it.

I searched for how to write an llms.txt file while writing this, eleven days after v2 landed, and nothing on the first page of results mentions v2. Several of them carry 2026 in the title and were written this year. One from 6 May still tells you the Optional block is "reserved for lower-priority content the parser can safely skip when context windows are tight". That is the first of the two things that stopped being true.

Two things you were told that are no longer true

The Optional section does not mean what you think

v1 came with a tool called llms_txt2ctx that expanded an llms.txt file into a context for a model, and the Optional section was an instruction to that tool: these are the links you drop when the context has to be shorter. That's where "put your secondary links under Optional and agents will skip them" comes from. The tool is no longer part of the proposal, and the meaning left with it.

Verbatim from the changes page:

The context-expansion tooling is no longer part of the proposal, and with it
goes the special meaning of the Optional section, which told those tools
what to omit. Optional sections are still allowed, and remain a useful
convention for secondary links, but they no longer carry mechanical
semantics.

So if you structured your file around Optional because a guide told you agents skip it, they never did. The skipping was one local tool's behaviour and never a promise any agent made, and now it's not in the spec either. It's a name for a section, a useful one that the spec still recommends, but only a name.

In my opinion this is barely a loss and mostly a correction. A mechanical Optional was only ever real inside llms_txt2ctx, and I've never seen an agent read that heading and act on it, so v2 removing the semantics changes what your file means on paper and nothing about what happens to it in practice. The thing left over is worth saying out loud though: there is now no way at all to tell an agent "this part is skippable", and that was a genuinely useful thing to be able to say.

"Zero or more" means zero

v2's Format section lists the parts of the file, and the H2 file lists are "Zero or more markdown sections delimited by H2 headers". The H1 is the one thing that has to be there, and the spec says so directly: it is "the only required section". So a file that is an H1, a blockquote summary and a paragraph of context, with no H2 anywhere in it, is a conformant llms.txt.

Plenty of tooling reports that as a spec violation. Mine did. The validator I put up on 12 August, two days after v2 landed, flagged "No H2 sections" with its basis set to spec, and in the same run it told you the Optional section was reserved by the spec for secondary links. Both of those were wrong on the day they shipped, and both were fixed on the 16th. A file with no H2 sections is still a file an agent can do nothing with, so it's worth a warning, but the warning is about practice and not about the spec, and labelling it wrong is how you get people restructuring files that were fine.

What v2 actually adds

Link relations for discovery. This is the one the changes page says people asked for most. Given a page, how does a client find its markdown version, or the llms.txt that covers it, without guessing at URLs? v2 answers with standard relations:

Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"

rel="alternate" type="text/markdown" points at a page's markdown version and rel="describedby" points at the llms.txt file that covers it. The clause worth reading twice is that these "can be provided as HTML <link> elements, or as an HTTP Link: response header. The header form also works for non-HTML resources, such as the markdown files themselves, and can be added in web server or CDN configuration without modifying any pages." So if you run a docs site behind a CDN you can ship the whole discovery half of v2 without touching a single page or redeploying anything.

Both markdown-twin URL forms are legal. v1 allowed only page.html.md, appending .md to the full page URL. v2 also allows page.md with the extension replaced, plus index.html.md or index.md for URLs without a filename. Publishing tools were already emitting the second form, and the spec blessed it rather than fighting it.

Subpath files finally mean something. v1 let you put an llms.txt at any path and said nothing about what that path implied. v2 does:

A file covers the URLs under its path, and where more than one file applies,
agents should use the most specific one.

That's also what lets you take part when you control a path and not the origin root. A GitHub Pages project site can never write into the host's /.well-known/, but it can put an llms.txt in its own directory and have that mean something.

Worth being precise about what that rule is and is not. It is not a search: an agent does not walk up the tree asking for files until one answers. Every file whose path is a prefix of the page's URL covers that page, and where several do, the longest path wins:

Mermaid source for the diagram

flowchart TB
    page["A page at<br>/workers/runtime-apis/"]
    other["A page at<br>/r2/buckets/"]
    root["/llms.txt<br>covers every URL on the host"]
    workers["/workers/llms.txt<br>covers /workers/ and below"]

    root -. "applies, but is less specific" .-&gt; page
    workers == "applies and is the most specific — an agent uses this one" ==&gt; page
    root == "the only file that applies" ==&gt; other

    classDef file fill:#dbe7fe,stroke:#2563eb,stroke-width:1px,color:#0f172a
    classDef url fill:#f2f5fa,stroke:#94a3b8,stroke-width:1px,color:#0f172a
    class root,workers file
    class page,other url

Both files genuinely cover /workers/runtime-apis/. The root one does not stop covering it because a deeper file exists — it is still the file that describes the rest of the host, and the deeper file wins only for the URLs beneath its own path.

Who actually does this

I checked these with curl on 21 August 2026:

Site	Root	Subpath
developers.cloudflare.com	200, 15,906 bytes	`/workers/llms.txt` 200, 82,671 bytes
docs.stripe.com	200, 89,857 bytes	—
docs.astro.build	404	—

Cloudflare is the one worth dwelling on. 15,906 bytes at the root and 82,671 bytes under /workers/ is the subpath rule doing its job in production: a small index for the whole developer site, and a big section-specific file for the part a coding agent is most likely to be deep inside already. Ask about Workers, get the Workers file, and the root file still covers everything the deeper one does not.

Here's what that pair looks like from the inside, checked from the page itself:

The note tagged spec is the v2 rule stated back at you: the file at /workers/ covers the URLs under /workers/, the root file covers everything, and an agent standing on this page takes the more specific one. Which file wins is a question you can only ask from a page, because the answer depends on which page you are standing on.

One curiosity while you're in there: llmstxt.org serves its own /llms.txt as text/plain. The spec says nothing about content types so nothing is broken, but if your checker treats anything other than text/markdown as a failure, it's going to fail the spec's own file.

Astro removed theirs

Astro deleted its files in withastro/docs#13538, titled "[i18nIgnore] Remove llms.txt files", merged on 20 April 2026. llms-full.txt and llms-small.txt went, and docs.astro.build/llms.txt is a 404 today as well. The stated reason is the one that should worry you if you maintain a docs site:

we've seen little uptake recently in usage — these files get very little traffic

They chose to put the effort into their MCP server instead, "and perhaps in the future offer per-page Markdown content", then signed off with "AI trends move quickly". The PR also shows what the files were costing them, flagged as "anecdotal obviously": two Netlify builds, 5m 6s with the files and 4m 22s without.

Two dates matter here. The removal is April and v2 is August, so what Astro measured was v1, and v1 gave an agent no way to find the file other than guessing the URL. Discovery is the exact hole v2 spends most of its new text filling. So I wouldn't read "very little traffic" as a verdict on the idea. I read it as a verdict on a file nobody could find.

That half-sentence about per-page Markdown aged strangely well. Astro walked out in April saying what they'd rather have is Markdown per page. Four months later the headline additions in v2 are markdown twins per page, plus the link relations that make them findable.

What v2 still doesn't do, and I'm not sure a text file at a well-known path ever can, is give an agent a reason to prefer your llms.txt over just fetching the page. Astro's answer to that was MCP, where the agent is handed the tool rather than having to go looking for a file, and that's a stronger position than a filename convention has ever been. Both things can be true: v2 is a real improvement, and the harder question is still open.

What to do about it

None of this takes long. If your file has an Optional section that you picked for mechanical reasons, keep the name if you like it but stop expecting it to do anything. If your pages have markdown twins, add the relations: one <link rel="describedby"> and one <link rel="alternate" type="text/markdown">, or the same thing as a Link: header at the CDN if you'd rather not go near the templates. serhiichuk.dev emits the describedby one on every page. And if your docs live under a path, an llms.txt at that path now has a defined meaning, so a large section-specific file sitting next to a small root one is a shape the spec understands rather than a shape you're getting away with.

What none of it buys you is search ranking. No engine has said it uses llms.txt for anything, and anyone who tells you otherwise is guessing.

I keep a checker for all of this at serhiichuk.dev/tools/llms-txt — it validates against v2 and labels every finding as spec, convention or hygiene, so you can decide which ones you care about. There's a Chrome extension as well, which runs the same rules against the tab you're on. That one exists because half of v2 is discovery, and the link relations live on the page rather than in the file — so a checker that only ever fetches a URL is looking in the wrong place for them. It also reaches what a hosted checker cannot: a staging host, an intranet address, anything that is not on the open internet.

Building an Autonomous SRE Agent with Google ADK and the Antigravity SDK

Yurii Serhiichuk — Thu, 09 Jul 2026 16:58:44 +0000

One of the most stressful parts of being an on-call engineer is triaging a production incident in the middle of the night. Modern distributed systems amplify the pain with extra cognitive overload of, well, the distributed systems: logs scattered across dozens of microservices, traces potentially stored in multiple different locations and a dozen of observability tools and dashboards on hand. While this (usually) makes total sense for SREs to design and develop this the way it is (maybe because of the costs or being knowledgeable in this or that tool, or just optimizing for a particular concern), this adds a lot of load for any on-call engineer.

So in era of AI, it makes total sense to get some aid from the autonomous and smart systems that are fine-tuned for your setup, that remember all the bits and pieces that are suitable for your infrastructure.

In this post I'm trying to go over a blueprint of such an autonomous SRE system - an AI agent that can be hosted near your main system and fine tuned to work with a set of tools you use.

I will be using Google Agent Development KIT (ADK) for multi-agent diagnostic reasoning and Google Antigravity SDK for the agent runtime - tool wiring, easy-to-apply safety policies and agentic orchestration. The setup will be deployed as a set of Google Cloud Run services but while being containerized can be easily ported to any other containerized platform. The entire stack runs locally as well with zero GCP credentials in a mock-telemetry mode, so you can try it in under a minute.

The Core Architecture: Reasoning + Safety

A proper SRE assistant has to get two things right at once: reasoning orchestration (which diagnostic step happens when) and environmental safety (the agent must never be able to mutate production while it pokes around). The blueprint splits these concerns across four small Fast API services that talk to each other over Agent-To-Agent protocol with results being streamed back as Server-Sent Events (SSE).

I'm using Google Antigravity SDK as an orchestrator and front-facing agent while it provides superior safety gates out of the box and is smart-enough to handle user requests.

safety_policies = [
    deny("*"),  # nothing is allowed by default
    allow("diagnose_sre")  # …except delegating to the SRE sub-agent
]

The config above basically gates whatever tools we want to allow for the agent to use.

And Google ADK agents fulfill the diagnostic workflows. ADK v2 which I am using is graph-based and the SRE sub-agent runs two-node graph:

TraceAnalyzer - scans recent trace summaries, filters for transactions that errrored or breached the latency budget (>5000ms), and isolates the single failing traceId.
LogCorrelator pulls the spans and the logs tagged with that traceId, then reasons over them with metric queries, cascade analysis, and post-mortem generation tools to produce the root-case report.

The third agent is an inventory agent that aids the SRE with aggregated knowledge of resources available in the GCP project. This helps grounding diagnostics and makes sure the agent works against particular resources instead of trying to look around too much.

Anatomy of a Diagnosis

When an alert fires, here's what actually is going to happen end to end from the on-call prompt to a finished post-mortem.

The Orchestrator does the one thing its policy allows: it hands the problem to the SRE sub-agent and steps back. From there the sub-agent streams its progress back as Server-Sent Events, so the on-call engineer watches the investigation unfold live instead of waiting on a spinner.

It starts by pulling the project topology — which services exist, how they call each other, and where their databases sit — from the Inventory agent's Firestore-backed cache; that map tells the diagnosis where to look. It then fetches the recent traces and runs them through the two-node graph: TraceAnalyzer collapses thousands of spans down to the single failing trace worth investigating, and LogCorrelator pulls every span and log line sharing that trace's ID to name the actual root cause — not just "the database was slow," but which span failed, with which error, and why.

Only then does it run the cascade analysis and draft the post-mortem, streaming the finished report back through the Orchestrator to the chat UI, where it lands as a rendered dashboard with a one-click download.

The beauty of such approach is extensibility of the setup. You can add new sub-agents or expand existing tools easily with ADK and A2A protocol.

Deep Dive: Cascade Latency & Bottleneck Analysis

In order to showcase the agent, I have also built another Fast API service that emulates real database errors with timeout exceptions and telemetry spans. As it is just a simulation, quite a lot of actual problems are quite similar to this one and frequently the initial gateways/backends latency or errors are hidden downstream. But of course this is a textbook example still: a gateway request that looks 10-second-slow, but where 99% of the time is actually trapped in a database call three levels down.

All three spans look ~10 s slow (inclusive), but the red /api/database bar is where the time is actually spent.

The cascade-analysis tool builds the span parent/child map and computes, for every span:

Inclusive duration — wall-clock time of the span including its children.
Exclusive (self) duration — the active time spent in that span alone:

ExclusiveTime(s) = InclusiveTime(s) - sum of InclusiveTime(children)

The span with the largest exclusive time is the true bottleneck. Here is the actual, verified output from incident simulator (available in the repo):

### 🔍 Span Latency Breakdown
Service / Span Name     Span ID             Parent ID           Status   Inclusive Time   Exclusive (Self) Time   Contribution
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
/api/gateway            span-gateway-111    None                ERROR    10270 ms         20 ms                   0.2%
  └── /api/backend      span-backend-222    span-gateway-111    ERROR    10250 ms         50 ms                   0.5%
      └── /api/database span-database-333   span-backend-222    ERROR    10200 ms         10200 ms                99.3%

### 🚨 Identified Bottleneck
*   Bottleneck Span:     /api/database (span-database-333)
*   Self-Execution Time: 10200 ms (99.3% of total trace)
*   Status:              ERROR
*   Error Message:       ConnectionTimeoutError: Failed to connect to db-primary.gcp.internal:5432 after 10000ms

The gateway and backend each look "slow" at 10 s inclusive, but their self time is a rounding error. The agent ignores the noise and points straight at /api/database — 99.3% of the budget, burned in a single connection timeout.

Automated Post-Mortem & One-Click Export

Diagnosis is only half the job; the deliverable on-call engineers actually need is a post-mortem. After the cascade analysis, a post-mortem generator drafts a complete Markdown document with an Incident Overview (time, root service, Trace ID, impact duration), a Timeline, a Root Cause Analysis, and a Prevention Plan — all populated from the real trace and log data.

That Markdown is then rendered via A2UI protocol

From SRE report markdown to a one-click, client-side post-mortem download.

A server-side translator spots the post-mortem heading and appends a download-button component; the browser renders it as a styled button that builds the file entirely in the browser — no server round-trip — from the Markdown it already holds.

One click exports post_mortem.md, ready to drop into your incident-review wiki.

And here is how this actually looks like in the deployed UI.

Least-Privilege IAM on Cloud Run

Handing an autonomous agent unrestricted cloud access is a non-starter. The deploy pipeline gives each Cloud Run service its own service account with the narrowest possible role set:

The write-only target app and the read-only SRE agent can never act on each other's plane.

The split is the whole point: the app that generates the chaos can only ever write telemetry, and the agent that investigates it can only ever read. Neither can act on the other's plane. (The deploy also provisions an inventory-agent-sa for topology discovery and a dedicated sre-build-sa for Cloud Build, each similarly scoped.)

Try It Yourself in 60 Seconds

You can run the entire scan → correlate → analyze → post-mortem loop locally. No GCP account or credentials required.

# 1. Clone the repo
git clone https://github.com/xSAVIKx/sre-agent.git
cd sre-agent

# 2. Install uv and sync the workspace (app, agent, sre_agent, inventory_agent, sre_common)
pip install uv
uv sync --all-packages

# 3. Run the incident simulation
uv run simulate_incident.py

The simulation triggers a database-timeout incident in the target app, writes mock traces and logs to mock_telemetry_data/, boots the Orchestrator in mock mode, runs the diagnostic workflow in-process, and prints the report to your terminal. You will see the structured telemetry logs (gateway → backend → database, ending in a CRITICAL ConnectionTimeoutError) followed by the full diagnosis: the 99.3% /api/database bottleneck table and the complete # 🚨 Incident Post-Mortem shown above.

Want the full multi-service experience with the web chat UI? docker-compose up --build brings up the Orchestrator, SRE agent, Inventory agent, a Firestore emulator, and the target app together — then open the chat at /chat.

Project Resources

The complete, runnable source — plus a step-by-step CODELAB that builds this from scratch — lives on GitHub:

👉 github.com/xSAVIKx/sre-agent

Open Knowledge Format - portable digital map of your data as code

Yurii Serhiichuk — Mon, 15 Jun 2026 07:38:08 +0000

Yesterday Google dropped an article describing an interesting idea - a portable format for knowledge sharing as markdown. This is not new or unique (llm wiki, obsidian values and even agents/claude.md are similar concepts). What stands out here is the openness of the proposed standard (also, just a v0.1 draft now) and actual ability of Google to make it a de-facto standard.

I've spent quite some time as GDE and AI integrator helping teams wire data into the LLM agents, and I keep hitting the same wall. The model is brilliant and the data is right there, but the knowledge about the data - what a table means, how a metric is defined or why a column was deprecated is, unfortunately, scattered across hundreds of different places. Something is available in company google docs or drive, the other things are available directly in readme and the extra missing piece is actually in that one engineer's head. Unfortunately, we can't write a connector for the last piece, but as Sam McVeety and Amir Hormati highlights in the blogpost we can at least help the agents (and developers) navigate through such knowledge better by introducing some shared and well known way of sharing the knowledge. And don't get me wrong, almost every team has probably already solved this in that or another proprietary way, rebuilding that same plumbing again and again from scratch.

OKF's wager against that mess is almost provocatively simple. If you can cat a file, you can read it. If you can git clone a repository, you can ship it. The spec only enforces one field to be explicitly provided - type , and that's exactly the trick: standardize only the smallest possible interoperability surface while leaving everything else to the people producing the data.

The reference repository indeed provides only the bare minimum - a pretty simple spec, example agent (which is BigQuery-oriented and pretty opinionated) and some sample bundles. And while this is a great starter pack, the idea of OKF how I see it is that knowledge lives everywhere. My SQLite file has knowledge. My Postgres instance has knowledge. My ~/notes folder and my Git history as well. If OKF is going to be a lingua franca, it needs producers for the messy real data the most of us actually work with, and it needs consumers that are not tied to any one vendor's agent.

So I built that: the OKF Skills, a vendor-neutral fleet of producers and consumers for OKF.

The idea is simple - any source in, any agent out.

What I built already is just a small baseline that I hope would be useful moving forward. That's six connectors - SQLite, MySQL, PostgreSQL, BigQuery, the local file system and Git. Each is a single portable Go binary with the same three commands: produce a bundle from the source, ingest a bundle to verify it (or --sync curated descriptions back to the source), and schema to describe itself. The SQL connectors can embed a per-column data profile and sample rows right in the concept docs.

Then three guidance skills with zero extra runtime. reader teaches an agent how to traverse any OKF bundle cheaply (index first, frontmatter only, grep before you read). The enricher teaches your harness to write good descriptions grounded in the schema, profile, and samples. And finally the producer-generator to bootstrap other connectors real fast.

I also decided to rebuild the visualizer Google team embedded into their agent. The viz renders any bundle into a single self-contained index.html near your OKF data. It produces a three-pane explorer with an interactive graph, navigator and content reader. Now there's no need to reach for Google-provided agent just to create a visualization if you just want that.

Finally, there's the mcp server that is capable of discovering every installed connector and exposes them as relevant tools. Point Claude Code or Antigravity at it and you're ready to rock. You can install all of them today with:

npx skills add xSAVIKx/okf-skills

Now you should be able to just ask your harness nicely to create, enrich and visualize the OKF bundle.

I hope this agentic tooling would allow the community to embrace the new format and truly make it a widely used standard. And also warmly welcome any new contributions to the existing connectors.

Useful links: