Making the Context Across 46 Repositories Semantically Searchable for AI

AI assistance disclosure: This article was drafted with the help of Claude. All technical content, design decisions, code references, and screenshots reflect production systems I designed and operate at airCloset; the prose was revised by me prior to publication.

Hi, I'm Ryan, CTO at airCloset.

In Part 1, I wrote about unifying 46 repositories of production code into a single knowledge graph via static analysis. The graph itself got built, but I closed the post with four open issues: no semantic search, node explosion, having to open the file to actually know what a function does, and the cost of writing a new parser every time a new boundary pattern showed up.

This Part 2 is about how I solved the first one — the entry-point problem (no semantic search). The other three are left exactly as Part 1 described them — I'll come back to them at the end, together with the new issues that surfaced once the entry-point problem was out of the way.

The reason to start with the entry-point problem is simple: if the graph exists but the only way to reach it is grep, the model ends up inferring anyway. The whole point — "give the model verified facts, not inference" — falls apart. So the entry-point problem had to be solved before the others.

The Hint Was in db-graph

Months earlier, I'd already solved the same structural problem in a different domain — the db-graph project.

Internally, we had a large number of DB tables spread across many services, and no single person had the full picture. Different people knew different pieces well, but the whole map didn't fit in anyone's head. So I built db-graph: extract schemas statically from ORM definitions, generate per-table descriptions with Gemini, embed them as 768-dimensional vectors in the graph, and make the whole thing semantically searchable in natural language.

At the time of that article it covered 991 tables. Today it spans 21 schemas / 1,133 tables / 10,815 columns, and finding data in natural language without knowing table names is just how people work now.

The pattern that proved out there:

Static-analysis graph + AI-generated context = natural-language semantic search works.

Bringing the Same Pattern to code-graph

If it worked for db-graph, it should work for code-graph. The moment that thought landed, I noticed something:

code-graph already contains "DB table nodes" as boundary nodes — they're one of the boundary node types I covered in Part 1.

So if I just join code-graph and db-graph, code-graph automatically inherits db-graph's semantic context. Without writing a single annotation, the existing assets alone make the graph meaningfully richer.

That's where the idea of "joining graphs" first came up — not treating each graph as its own island, but designing the joins between them.

But API / Event / Page Still Need Meaning — and Annotating Every Function Is Off the Table

Joining db-graph took care of DB context. But the remaining boundaries (API / Event) and the graph's entry-point type (Page) still need meaning attached. Static analysis alone can't pull intent out of those, so context has to come from somewhere else.

The choice was clear: write the intent directly into the code via annotations (the same approach used by cortex's internal knowledge graph, which I covered in AI Harness Series, Part 2).

The catch: you can't annotate all the functions across 46 repos. There must be tens of thousands of them. Asking established teams running an existing production codebase to retroactively annotate everything is just not realistic.

But here's the second realization:

What matters is just the boundary nodes. So if I only annotate around the boundaries, that's enough.

When an AI agent asks "what breaks if I change this code" or "what other repos call this API," what it needs isn't a per-function logic explanation. It needs boundary intent — what is this screen for, what does this API return, what milestone in the business does this Event mark.

= Minimum annotations, maximum meaning. That became the heart of the design.

Designing the annotation graph

Putting it together (internally we call this annotation graph service-product-graph, or SPG):

Three graphs sit as peers, joined by SAME_ENTITY edges. There's no hierarchy — you can start from any graph and reach the others.

• code-graph (structure) — functions / classes / boundary nodes from static analysis (46 repos)
• db-graph (DB context) — 1,133 tables, semantically described
• annotation graph (intent) — @graph-* tags written only around boundaries

The entry point for AI agents is a single MCP server that traverses all three graphs. AI agents never hit db-graph directly — the annotation graph's MCP server proxies db-graph calls on their behalf.

The annotation graph has 7 node types: Page / Section / Dialog / Field / Action / Api / Task. The early version was screen-focused and called screen-graph, but once it grew to cover backend Api / Task, it was renamed to service-product-graph.

An Annotation Example

Here's what an annotation looks like (fictional, but close in shape to the real ones):

/**
 * @graph-page /home
 * @graph-business Main screen. Members can see what they're currently renting, buy items, and initiate returns.
 * @graph-label Home Screen
 * @graph-has-section banners, wearing-items, wearing-return, delivery-status
 * @graph-has-dialog buying-modal, return-modal
 * @graph-navigates-to /return-procedure, /checkout, /my-karte
 * @graph-calls GET /api/v1/wearing
 * @graph-reads admin_delivery_orders, admin_rental_items
 * @graph-flow styling-loop
 * @graph-status monthly-member
 */

Two things matter here:

• @graph-business carries the intent text (in our actual codebase it's written in Japanese). This is exactly what gets vectorized — it's the substance of semantic search.
• @graph-flow / @graph-status carry where this sits in the member lifecycle (free signup → monthly subscription → styling loop → cancellation, etc.) and which member segment it's for. They add a second dimension of meaning: "this screen shows up inside the styling loop for monthly members."

There's also @graph-case (the conditional pattern tag that test cases derive from), but that's for another time.

Running Annotations Without Interfering With the Day-to-Day Dev Workflow

This is where it gets practical.

Once I committed to building annotation graph, here were the constraints:

• Engineers run normal product dev with human code review
• AI review isn't wired up on every repo yet — cortex's fully automated review (covered in AI Harness Series, Part 6) only works inside the cortex monorepo
• Asking humans to review annotations on top of their normal review load is a non-starter
• Even a split like "humans review the code, the AI reviews the annotations" inside the same PR mixes two review streams together and just confuses everyone

In other words: don't mix humans and AI inside the same PR.

The solution was to physically separate annotations onto their own branch.

• Leave main untouched; engineers' normal flow stays exactly as it was
• Stand up a separate annotation branch that's the AI's exclusive territory
• When main changes, a webhook fires
• The annotation branch handles generating the diff annotations and reviewing them — the AI does both, end-to-end
• From the engineer's side, they only touch main and don't even need to know annotations exist

This is the "every line of code passes through an AI gate" ideal from AI Harness Series, Part 6, adapted to the constraints of an existing organization. cortex (the internal AI platform) is a monorepo I assemble from scratch, so "every commit passes the AI gate" actually holds there. For the 46-repo production system, that precondition doesn't hold. So instead of giving up on the ideal, I split it: engineers' workflow on one branch, AI's annotation workflow on another, both running in parallel.

Protecting Cross-Graph Consistency With an SLO

Just running the annotation pipeline doesn't guarantee the quality of the joins between the three graphs (code-graph / db-graph / annotation graph). So there's a set of SLOs that automatically check the consistency across the entire graph.

The main rules:

• API chain connectivity — at least 95% of HANDLES_API handlers must have downstream function calls (= no handlers that receive an API and then do nothing)
• DB access completeness — at least 80% of DB read/write edges must be joined to db-graph column nodes (= code-graph's DB boundaries are connected to db-graph's meaning)
• Event field resolution — at least 70% of Event edges must carry field-level information
• No ambiguous edges — name-resolution-ambiguous edges must be 0 (severity: error)

These are really just a naive question — "shouldn't the boundaries connect to each other?" — turned into an SLO. If anything drops below threshold, an alert fires, and the trustworthiness of the whole graph gets defended every day.

The daily boundary-analysis cron from Part 1 (5% connection-rate drop = alert) was code-graph-only. This is a cross-graph SLO — it guards the joins between graphs themselves. Add a parser to one repo, write a new annotation, change a schema — whatever happens, by the next morning a quality drop in any join becomes visible.

Joining the Static Graph and the Annotation Graph via SAME_ENTITY Bridges

I've been writing "join" casually, but the actual joining wasn't that straightforward.

Static-analysis API / Page / Task nodes and annotation graph API / Page / Task nodes are created as separate nodes. They mean the same thing, but their names / paths / identifiers don't match by themselves — there's nothing automatic about lining them up.

To connect them, we generate a separate edge type called SAME_ENTITY. There are three bridges:

• API bridge — API path normalization with a 4-stage fallback
  1. Per-repo prefix conversion (e.g., normalize console-side /console/api/ to /api/)
  2. Version stripping (/v1.x/ → /)
  3. Parameter normalization (unify /:id, /{id} to /:dynamic)
  4. Exact match → tolerate trailing ? → strip trailing :dynamic? → finally fall back to a dynamic-dispatch boundary :dynamic, loosening progressively
• Page bridge — 6 strategies applied in priority order (URL direct match, component path match, itemId match, PascalCase normalization match, parent-directory linking, strip dynamic segments and match parent URL)
• Task bridge — 8 per-repo patterns

There was also one operational footgun. The first implementation used INSERT NOT EXISTS to avoid duplicates. But BigQuery's streaming-buffer visibility lag let duplicates slip in — in one repo the edges doubled from 106 to 214 overnight. We fixed it by rewriting to MERGE INTO to make the operation idempotent.

The Result: Entering the Graph from "the subscription-fee calculation"

With all of this in place, the entry-point problem from the end of Part 1 was finally solved:

"the subscription-fee calculation for members seems off"

Throw this natural-language query at annotation graph and vector search returns the related nodes (Page / Api / Function / DB table) as facts. From there, SAME_ENTITY takes you over to code-graph functions, including callers and callees in other repos. From the DB boundaries in code-graph, you can cross into db-graph and pull the relevant columns.

The entry point can be anywhere — "what calls this table?" starts from db-graph, "what's the blast radius of this function?" starts from code-graph, both walk the same connected network. From a single natural-language query, or from a specific node, you can now traverse all three graphs and get every relevant piece of code plus every relevant DB schema.

The Part 1 lament — "the graph is there but the entry point is missing" — could finally be put to bed.

Real Usage Numbers

From 2026-04-16 (first production deployment) to the time of writing — about 2.5 months — the annotation graph's MCP server has handled ~50,000 calls from ~73 users. The breakdown:

• Engineers (PI Division + QA + relevant engineering teams) — ~47,000 calls / 51 users
• Non-engineers (stylists / customer support / mall operations / executives / administration) — ~2,800 calls / 21 users

The interesting line is the second one. "Search the codebase in natural language" is usually an engineer's tool — but once the entry-point problem was solved, people outside engineering started using it too, asking things like "how does this feature actually work?" or "what's in this DB?" in their own words.

This is adjacent to the "non-engineers writing specs with AI" trend I covered in AI Harness Series, Part 5 — a graph that can be queried by meaning starts to matter org-wide. Call volume is overwhelmingly dominated by engineers, of course. The interesting thing is the range of job roles starting to pick it up. That's the real impact of solving the entry-point problem.

MCP as the Single Front Door

The MCP server is the cross-graph entry point. It exposes six tools — service search / service detail / API detail / data-flow tracing / impact-radius tracing / business-rule full-text search — and that's the only entry point AI agents ever touch.

One design choice worth calling out: AI agents never talk to db-graph directly. The annotation graph's MCP proxies db-graph calls. From the agent's side, the mental model stays simple: "ask one MCP and get everything back."

That makes the full chain — "Screen → API → Code → DB → Column" — traversable in a single MCP tool call.

April–May Timeline of Trial and Error

Same approach as Part 1 (pulling commits from Jan–Mar). For Part 2, the key commits are from April–May.

April: Expansion and the First Bridges

• 2026-04-14 ─ refactor(graph): rename screen-graph to service-product-graph — declaration that the scope expands from screen-only to whole-service
• 2026-04-15 ─ feat(graph): add Api and Task node types to service-product-graph parser — Api / Task node types added
• 2026-04-15 ─ feat(mcp): add cross-graph tools to service-product-graph MCP — cross-graph tools land (the single front door across all three graphs)
• 2026-04-15 ─ feat(graph): add SAME_ENTITY bridge edges between service-product-graph and code-graph — first bridges
• 2026-04-18 ─ feat(graph): resolve Redis keys to code-graph boundary nodes — boundary resolution through Redis
• 2026-04-19 ─ feat(service-product-graph): add EventBridge EMITS_TO support + SAME_ENTITY bridge
• 2026-04-20 ─ feat(code-graph, service-product-graph): improve SAME_ENTITY boundary bridge coverage — 4-stage fallback locked in
• 2026-04-21 ─ feat(auto-review): SPG annotation auto-maintenance pipeline — AI auto-maintenance pipeline (= what Part 1 hinted at with "humans alone can't, but AI can")
• 2026-04-22 ─ feat(service-product-graph): add Task SAME_ENTITY bridge to code-graph — all three bridges in place

May: Stabilizing and Expanding

• 2026-05-01 ─ Annotation generation moves from local execution to a Cloud Run Job; operation stabilizes
• 2026-05-05 ─ feat(spg): add mall repos to SPG indexing — mall repos indexed
• 2026-05-06 ─ feat(spg): add Go-aware parser — Go support
• 2026-05-06 to 08 ─ Page bridge strategies expanded to six, connection rate hits 100%

What This Timeline Says

April 15 was the day "expansion + cross-graph tools + bridges" landed in close succession. Over the next week, "Redis / EventBridge / Task bridges / annotation auto-maintenance" stacked up week over week.

In particular, the annotation auto-maintenance pipeline on April 21 is where the "humans alone can't do this, but AI can" promise from Part 1 got cashed in. From that point on, annotation shifted from "humans grind through writing them" to "design the whole operation assuming AI writes them."

What Still Isn't Solved

Solving the entry-point problem didn't make everything clean. A few issues remain.

1. Maintaining Annotation Coverage

The frontend side is annotated heavily. Backend / Go / batch are still thin. Some nodes will always be missing annotations — that's structural, and you can't drive it to zero. It's an ongoing operational issue.

2. Bridge Mis-Joins Aren't Fully Eliminated Structurally

The Page bridge in particular has cases where multiple annotation Pages map to the same boundary — that's structural and unavoidable. Adding more strategies got coverage to 100%, but guaranteeing "every join is correct" 100% is hard.

3. No Dynamic Analysis

The graph only carries the fact that "this edge exists statically." How often that edge actually gets used in production isn't recorded. Piping production execution counts back into the static graph and surfacing dead-code edges as a separate signal — that's still untouched.

4. Onboarding Cost When a New Repo Joins Production

Every time a new repo enters production, the bridge normalization rules and per-repo patterns need adjusting. This is the annotation-graph-side version of Part 1's fourth issue (the cost of adding a new parser for every new boundary pattern).

Closing: Not "Thrown Away," but "Evolved"

In Part 1's closing note, I touched on the fact that the cortex side (the internal AI platform) bailed out of the code-graph approach early and bet on an annotation-based knowledge graph instead. The bail-out was fast enough that calling it "thrown away" wouldn't be wrong — but looking back across this whole series, the more accurate word is "evolved."

What it evolved into, in the end, is three graphs joined as peers:

• code-graph (structure)
• db-graph (DB context)
• annotation graph (boundary intent)

Joined by SAME_ENTITY, served to the agent through MCP. The thing static analysis alone couldn't deliver — querying by meaning — became workable by reusing the db-graph success pattern and adding minimal annotations only at the boundaries.

And one more framing: paired with the AI Harness Series, Parts 1–6, this series sits as:

• AI Harness series — how to live with AI when you're assembling the system from scratch yourself
• code-graph-deep-dive series (Part 1 + Part 2) — how to live with AI inside an existing organization's running production system

= the same philosophy (design without trusting AI), implemented under two different sets of constraints.

Thanks for reading this far.

Comments

[Wren Calloway]

@ryantsuji The exact-equality-after-transformation point is the correction that matters, and I want to make sure I absorb it rather than nod past it. You're right that "the transformation pointed at a different real entity than the author intended" is a categorically different animal than "the scorer picked a similar-looking node." My original framing smuggled in the fuzzy-match failure mode without earning it. Granularity loss is the real shape here, and it's actually the more insidious one to alert on precisely because every individual bridge is defensible in isolation — the boundary does contain the component. There's no single wrong edge to point at, just a distribution that's quietly coarser than the annotation author's mental model.

Which makes me think the drift alert wants a second signal beyond "% resolving via strategies 3–6." That catches the cascade sliding down the priority order, but the parent_dir absorption you're describing might not move that number much if strategy 5 was always doing the work — it'd show up instead as fan-in: multiple SPG nodes converging on one boundary. So maybe the thing to threshold is the ratio of collapsed nodes per boundary per repo, not just which strategy won. A boundary that suddenly absorbs six Pages where it used to hold two is the granularity-loss tell even when match_strategy never changed.

The daily cron diffing stored bridges against a from-scratch re-resolve is the piece I'd underweighted, and it's the closest thing you have to a correctness guard that isn't a human noticing. The gap is that it catches strategy flips, not stable-but-coarse joins that never flip. Pairing that with the quarterly owner-sampling is right — owners are the only ones who actually hold the intended granularity, and no cron recovers that from the derived identifiers alone. Good thread. I learned the shape of my own critique better by being wrong about half of it.

[Ryosuke Tsuji]

The fan-in signal is the piece I'd been missing. You're right that strategy-drift alone misses stable-but-coarse joins, because "strategy 5 has always won" won't move the distribution while the number of SPG nodes pointing at a single boundary quietly climbs. Ratio of collapsed nodes per boundary per repo, tracked over time with a delta threshold, is a sharper monitor. It catches the failure mode where every individual bridge is defensible but the aggregate is coarser than the author intended.

One layer I'd add on top: the same signal has a runtime use, not just a monitoring use. When an agent queries a boundary that aggregates N SPG Pages, that N itself is a confidence hint the agent should surface ("this boundary corresponds to N distinct annotated pages, treat the granularity accordingly"). Same information, shifted from "we notice granularity drift once a week" to "the consumer sees the granularity every query." Neither is implemented, but the second one might actually change AI behavior in practice, since the LLM in the loop can be told to treat high-fan-in boundaries as ambiguous rather than definitive.

The gap in the cron you identified is exactly right. It's a strategy- flip detector, not a granularity-drift detector. That's a good reminder that "we already have a monitor for that" and "we have a monitor for the specific failure mode you're pointing at" aren't the same thing, and I probably lean on the cron too hard as a catch-all mental model.

Owner-sampling as the correctness backbone is where this lands for me too. Deterministic identifiers can encode "which entity does this transformation point at" but they can't encode "what granularity did the author actually want." Only the owner holds that. The sampling is expensive compared to a cron, but it's the only piece that closes the loop on intent-vs-derivation. Combined with fan-in ratio alerts, that's a stack that starts to look like it's guarding correctness rather than existence.

"I learned the shape of my own critique better by being wrong about half of it." That's the read I aspire to when I comment on other people's technical work. Really appreciate it.

[Wren Calloway]

The SAME_ENTITY bridge coverage hitting 100% is the number I'd worry about most, precisely because you got there by adding strategies. Your issue #2 admits the Page bridge maps multiple annotation Pages to one boundary — which means "100% connected" and "100% correctly connected" have quietly diverged, and once they diverge the SLO stops measuring what it looks like it measures. A HANDLES_API handler can be 95%-connected downstream and still be bridged to the wrong annotation, and the vector search will happily return that wrong node as a "verified fact." That's a nastier failure than grep-inference, because it wears the costume of ground truth.

The thing your SLOs guard is edge existence; what you can't cheaply guard is edge correctness, and the 6-strategy priority order is exactly where a silent mis-join hides — strategy 5 (parent-directory linking) firing when strategy 1 should have. If you're not already, I'd track how often the winning bridge is a low-priority fallback per repo and alert on drift there, not just on connection rate. A repo where suddenly 30% of Page joins resolve via "strip dynamic segments and match parent" is telling you the confident-but-wrong rate is climbing even while your dashboard stays green.

One genuine question, not a gotcha: when a mis-join surfaces, does an agent or a human catch it, and how? That correction loop is the actual trust boundary of the whole system, and it's the one piece the post doesn't show.

[Ryosuke Tsuji]

The 6-strategy priority order framing is accurate for the Page bridge.　That's literally what the code does: candidates are UNION'd across six　matching strategies (url_pattern / component_path / item_id / url_pascal　/ parent_dir / boundary_dynamic_strip), ranked by strategy priority,　top-1 wins per SPG node. The Api bridge has the same shape with four　stages (exact normalized match, trailing ? variance, trailing　/:dynamic strip, boundary :dynamic regex against literal segments for　dynamic dispatch). So the cascade isn't unique to Page.

One nuance that changes the failure-mode profile though: each strategy　still resolves to exact equality on a deterministically-derived　identifier. No fuzzy string matching, no edit distance, no substring　scoring. The "heuristic" part is the choice of transformation rule　(strip this prefix, PascalCase these segments, walk up to parent　directory). Once the transformation runs, the actual join is =. That　constrains how mis-joins can happen: not "the algorithm picked a　similar-looking wrong entity" but "this transformation rule pointed at a　different real entity than the annotation author intended."

The mis-join mode we actually see (issue #2) is the second kind, and　it's granularity loss rather than confidently-wrong: multiple SPG Pages　collapse onto one boundary when a shared parent directory catches them　(strategy 5, parent_dir). Each individual bridge is "correct" in the　sense that the boundary does contain that component, but the annotation　author often wanted more granular boundaries than exist. You're right　that this still isn't what a "% connected" SLO catches, so your critique　lands, just aimed at collapsed granularity rather than plausible-but-　wrong.

Your alerting suggestion is the right shape for detecting that.　Per-strategy tagging is present on each edge (the match_strategy field　is stored on the SAME_ENTITY row), so per-repo distribution is queryable　retroactively. What isn't wired up is the drift alert. Threshold on　"% of joins resolving via strategies 3 through 6 per repo per week" and　pager on delta would surface the parent_dir absorption pattern you're　pointing at. Not implemented today.

To your genuine question on correction loops. Mis-joins currently　surface through three paths:

• An engineer notices during use (agent returns a plausibly wrong　neighbor, they dig in)
• The daily boundary-analysis cron re-runs the strategy resolver from　scratch and diffs against stored bridges; a bridge that flipped　strategies shows up in the change log
• A downstream test fails (rare, because most consumers are LLMs that　don't verify)

Human is the primary catcher today. AI in the loop is good at using　bridges but bad at questioning them, precisely because they look like　ground truth. Making the correction loop first-class rather than an　artifact of use is the actual gap you're identifying. The owner-sampling　proposal from an earlier thread in these comments (a quarterly sample of　~50 bridges per relationship type, marked mismatch by entity owners) is　aimed at exactly this. Combined with your strategy-drift alerting, that　gets a lot closer to guarding correctness rather than just connectivity.

"Confident-but-wrong wearing the costume of ground truth" is a phrase　I'll be borrowing. Thanks for the read.

[Eryc Tri Juni S]

The refusal architecture part of this thread really got me — coming at it from a different angle (public content/GEO, not internal code graphs), but I think I'm seeing the same thing. In my own RAG testing, when the model doesn't have enough retrieved context it almost never says so, it just... fills the gap and sounds confident about it anyway. Same root problem, just showing up in content retrieval instead of code search.

What stood out is you pushing for server-side gating instead of trusting the agent to flag its own uncertainty. Makes sense — if you leave it up to the model, it'll turn "not sure" into a fluent sentence every time, because that's what it's optimized to do.

Now I'm curious if something like that could work outside a controlled internal setup — like, could a public site expose some kind of confidence signal to crawlers, or does this only work because you control both sides of the pipeline?

[Ryosuke Tsuji]

Glad that part landed. Your RAG observation matches exactly what pushed us to server-side gating. The model turning "not enough context" into a fluent answer isn't a bug in any one model, it's what generation is. So the "I don't know" has to be produced by something that isn't a generator.

On your question, I think the honest answer is that the enforcement half only works because we control both sides. What the server-side gate actually does is refuse to hand the model ambiguous material in the first place. Below a similarity threshold it returns a typed "no confident match" instead of the top-N-whatever, so the model never gets the raw ingredients to hallucinate from. That refusal happens before generation, and it requires owning the retrieval server.

A public site can't enforce that, because the gate has to sit on the consumer's side of the pipe and you don't own the crawler. What a public site can do is make the honest path cheaper than the guessing path. Structured data, explicit scope statements ("this page covers X as of date Y, it does not cover Z"), stable anchors, machine-readable freshness. None of that forces a model to admit uncertainty, but it changes what retrieval hands the model, which is the same lever we pull internally, just without the guarantee.

For crawlers specifically, my guess is confidence signals end up like robots.txt. A convention that well-behaved consumers respect and nothing enforces. Publisher-supplied relevance signals also have a trust problem baked in, since they're exactly the kind of thing that gets gamed (meta keywords walked this road already).

There is one path where a publisher genuinely gets the gate back, though. Expose your content as an MCP server and let agents call it directly instead of scraping. Then you own the retrieval endpoint again, and everything we do internally becomes available to you. Typed refusals below a threshold, explicit scope answers, even query logs showing what AIs are actually asking about your content. Stripe and Cloudflare already do this for their docs, and coding agents really do call those endpoints. To be clear, this doesn't move the needle on how search engines rank or summarize you. It's a separate serving channel for the agentic side of consumption, not an SEO lever. But for the slice of traffic that comes from agents rather than crawlers, it's the one place the publisher controls both sides of the pipe.

[Eryc Tri Juni S]

Really appreciate you taking the time to lay that out — "refusal has to happen before generation, and that requires owning the retrieval server" clarified something I'd been fuzzy on.

To answer my own question with what I think the honest scope is: MCP is for agents that act — bidirectional, you own both ends, you can gate before generation. What I built (AGP) is aimed upstream of that, at crawlers that only ever read, once, passively, with no request/response loop to gate at all.

Mechanically: I picked Google Sites deliberately as a worst-case CMS — it can't produce clean canonical tags or structured data, and it's on a zero-authority .my.id domain, so if this works at all, it isn't explainable by domain trust or CMS quality. A Cloudflare Worker sits in front of it and uses HTMLRewriter to edit the HTML mid-flight, as it streams past — not replacing the origin, just correcting what it hands off. Crawlers get a separately-stored clean semantic payload injected in; humans get the untouched page. Same content, different container, no attempt at persuading anyone of anything.

So probably not competing solutions — MCP fixes the action side, AGP (at best) fixes whether the read side is even structurally intact in the first place. Still an open question whether "structurally clean" is enough on its own, or whether a crawler needs a way to signal back "was this actually sufficient" — which sounds like exactly the gap your server-side gating closes on your end.

[Ryosuke Tsuji]

The worst-case-CMS setup is good experimental design. Zero-authority domain plus a CMS that can't emit structured data means if retrieval quality improves, you've actually isolated the variable. Most GEO claims never control for domain trust, so this alone puts your test ahead of the field.

The part I'd watch is the crawler/human split itself. In classic SEO that shape has a name, cloaking, and engines penalize it regardless of intent because they can't verify intent, only divergence. Your defense is semantic equivalence, same content in a cleaner container, and I think that's the right line to hold. But it's worth designing as if someone will eventually check. If AGP-like layers spread, the obvious countermeasure for engines is to render both variants and diff the semantics, and the setups that survive that check are the ones where the equivalence is real. Yours sounds like it is. The gray zone arrives when people copy the architecture and start "improving" the crawler payload beyond what the human page says.

On the feedback question, you have more signal than you might think. You own the Worker, which means you own the access logs, and crawl patterns are a crude but real read on what retrieval is doing. Which paths get fetched, how often, whether a crawler comes back after the clean payload shipped. It's nowhere near a "was this sufficient" signal from the model side, but it's the same move we made internally, when you can't gate the consumer, instrument the part of the pipe you do own.

[Eryc Tri Juni S]

Went back and actually audited the code against your point rather than just agreeing with it. Two findings worth sharing.

First, the element-stripping for AI bots (script/style/iframe/header/footer) turned out to be justified, but I wanted to verify rather than assume: Google Sites ships large non-semantic JS blobs inside those wrapper tags, and the bot-facing payload carries its own replacement footer with the actual contact/nav info. So the stripping is noise reduction, not content removal — checked it, the semantic substance survives in the KV payload, the JS bloat doesn't. Good distinction to have confirmed rather than assumed.

Second, honestly, the audit surfaced that not every page in the architecture is at the same maturity level. The case study page holds up well on content parity. Older parts of the routing logic still have some rough edges I'm working through — exactly the kind of thing your diffing prediction would catch, which is the point. Better to find it now going page by page than have it found later.

Appreciate the push to actually check instead of just asserting it.

[Ryosuke Tsuji]

Glad the comment turned into an actual audit, that's the best outcome I could have hoped for from this thread. And finding the uneven pages yourself, before any crawler-side diff exists, is the whole game. Good luck with AGP, I'll be curious how the experiment reads in a few months.

[Eryc Tri Juni S]

Appreciate that, genuinely. This whole thread ended up being worth more than four months of trying to get anyone to look at it — the diffing point specifically is going to stick with me long after this conversation ends. Going to keep working through the rest of the pages the same way, on my own time now rather than in public. Good talking through this with someone who actually knows the terrain. Take care.

[Siyu]

The "entry-point problem" you articulate, where the knowledge graph exists but has no semantic surface for natural-language queries, maps directly onto a parallel problem in professional discovery. LinkedIn has the data, CVs have the data, but there is no way for an AI agent to ask "who in my network has led a database migration, communicates asynchronously, and prefers written briefs over meetings" and get verified results. I work on Opportunity Skill, which builds structured semantic impressions from daily work patterns that other agents can query, solving the same category of problem for professional knowledge instead of code knowledge. In both cases the cost is the same: the data already exists, but no one can reach it.

What interests me most about your annotation philosophy is the deliberate minimalism: only boundary nodes get annotated, and AI handles it entirely in a parallel branch with no human review. This is not a compromise, it is a design insight. Boundary nodes carry disproportionate meaning relative to their count. A single @graph-business tag on a page explains more about what the system does than annotating every internal function ever could. The parallel-branch decision goes further: it acknowledges that maintenance cost, not annotation quality, is the bottleneck at scale. Humans cannot be relied upon to keep annotations current across 46 repos because annotation drift is invisible to them. AI can, because annotation generation is just another pipeline step. The same principle applies to maintaining professional semantic profiles: humans forget to update their bios, but an agent observing daily work patterns can refresh impressions continuously without the user lifting a finger.

The non-engineer adoption data, 2,800 calls from stylists and customer support, reveals something beyond "semantic search works." A graph queried by meaning reallocates who gets to ask questions. When the access interface shifts from grep to natural language, knowledge stops belonging exclusively to the people who know where it lives and starts belonging to anyone who knows what they need. The same threshold exists in professional discovery: the barrier between needing a connection and finding one is not missing information, it is missing semantic entry points. The moment profiles become agent-queryable rather than human-readable, the entire cost structure of discovery changes. The graph was always there. The entry point was not.

[Ryosuke Tsuji]

This is the parallel I was hoping someone would see — and your cross-domain framing is sharper than what I'd been holding internally.

On boundary-node minimalism: yes, exactly. Once I accepted that internal helpers don't carry semantic weight (only the entry/exit surfaces do), the cost equation flipped. I'd been thinking about "how to annotate everything" when the answer was "annotate almost nothing, but pick the right almost-nothings." Boundary nodes are where intent is expressed because they're the only places where the system communicates with anything outside itself.

I'd extend your maintenance-cost framing by one layer: humans don't just forget — they actively don't notice drift, because the drift is between code and meaning, not between code and code. Compilers catch code-vs-code drift. Nothing catches code-vs-meaning drift except a verifier that also reads meaning, which, pre-LLM, was a human reviewer context-switching every PR. AI verifying code-vs-meaning at PR time, every PR, asymmetric cost-wise — that's the actual unlock.

The parallel-branch implementation is adaptation to the constraint we have today: 46 long-running repos where retrofitting annotations into main isn't realistic. For new environments we're building AI-native from day one, annotations sit in main alongside code, generated and reviewed in the same PR — convergent steady state, different starting point. The parallel-branch is the legacy path; main-branch annotations are the greenfield path.

The "who gets to ask questions" point is where my own framing was still incomplete. I'd been describing it as "non-engineers can search now," but yours is more accurate: the entry point is where access is gatekept. grep was a competency moat masquerading as a tool. The moment the interface becomes natural language, knowledge becomes available to whoever knows what they need. The graph was always there — the entry point wasn't.

Curious about Opportunity Skill. The piece I'd most want to understand: how do you handle freshness/staleness on observed-work impressions? Code annotations regenerate on PR diff, which is a discrete signal. Continuous work observation has no such discrete trigger, so I'd imagine you need a different mechanism for "this impression is current" vs "this is stale."

[Mike Czerwinski]

The recall math from Part 1 frames the accuracy ceiling at >99% per hop. Part 2's SLOs sit one layer under: they measure intra-graph connectivity (API chain >=95%, DB completeness >=80%, event fields >=70%, ambiguous = 0). Completeness, not correctness of the bridge itself.

Worth a second pass for one shape the SAME_ENTITY normalizations could be hiding. Recall counted at the edge measures edges that exist. It does not catch edges that connect what they should not. A PascalCase normalization with 99% recall can also produce a 5% wrong-pair join rate if the normalization over-fits on local naming. The recall number is intact. The graph's downstream truth is not.

The way to surface that error mode is an exogenous check the bridge author does not control: pull a sampled 50 SAME_ENTITY edges per quarter, hand to whoever actually owns the entities on each side (the API team, the page team, the task team), and have them mark mismatch. The cost is finite, the sample is small, and the SLO threshold becomes "% of sampled bridges confirmed correct by entity-owner >= 95%", distinct from "% of handlers reachable >= 95%". Same target, different layer of trust.

The 30% non-engineer usage from this part is the more telling receipt for the system than any of the connectivity numbers. Stylists, ops, executives querying natural language through MCP and trusting the answer means refusal architecture matters as much as connectivity. The question I keep landing on for systems at this scale: what does the MCP return when the graph genuinely does not know? A confident wrong answer at 95% connectivity is more dangerous than a low-confidence answer at 70%. Curious whether SPG has explicit "I don't know" semantics, or whether it always returns the closest annotation it can find.

[Ryosuke Tsuji]

Thanks for the careful read.

The framing I'd push back on first: those SLOs aren't measuring bridge correctness — they're connectivity floors, set deliberately conservative. We'd rather under-recall than produce false-positive joins, so the 95/80/70 isn't a ceiling we're chasing; it's a "don't let this regress" floor we won't drop below. Each SAME_ENTITY normalization is tiered exact-match-first with eager gating on the looser fallbacks. Chasing higher recall would mean loosening those gates, and the regression alarm would trip. A confidently wrong neighbor is worse than a missing one — especially with non-engineers in the consumer loop, where the cost of a wrong-but-plausible answer changes character entirely.

That said, the entity-owner sampling check you described is a layer we don't currently have, and the framing is clean: "% of sampled bridges confirmed by owner ≥ 95%" as a trust signal distinct from connectivity. Worth adding.

On "what does MCP return when the graph genuinely doesn't know" — semantic search returns ranked similarity scores, so the agent sees signal it could reason about, but explicit refusal isn't first-class yet. A real gap. Coverage SLOs guarantee that high-confidence neighbors exist when they should, but they aren't refusal signals on their own.

[Mike Czerwinski]

Recasting the SLO as a floor rather than a ceiling clears most of the confusion I was carrying into the read. A floor on connectivity says "do not regress below this, because below this the consumer reasoning collapses." A ceiling on correctness says "this is the truth we will defend." Those are completely different commitments, and conflating them is what makes recall-at-95 sound like a quality claim when it is actually a non-regression contract.

The owner-sampling check then sits cleanly orthogonal to the SLOs. SLOs guarantee neighbors exist; sampling tells you whether the ones that exist are the right ones. Reporting them separately keeps each one honest, because either can move while the other holds.

Refusal as first-class output is the part I would push hardest on next. Ranked similarity scores are useful debugging signal, but until the consumer agent can receive an explicit "the graph does not know this" the silent failure mode is plausible-but-wrong, which is exactly the failure consumers cannot detect. The SLOs you have today can promise that high-confidence neighbors exist when they should; they cannot promise that absence is communicated. Those are different guarantees and a non-engineering consumer needs both.

[Ryosuke Tsuji]

Recast nails it. Floor for non-regression, ceiling for truth, completely different commitments, and yes, "recall at 95" reads as a quality claim when the actual commitment is non-regression. The article didn't separate those carefully enough; that's on me.

Owner-sampling as a separate report line is going on the roadmap. Treating SLOs (neighbors exist) and owner-confirmed correctness (they're the right ones) as independent gauges that can drift independently is the right framing. The cost of running ~50 sampled edges per relationship type quarterly is bounded, and it gives entity owners a recurring stake in the bridges that point at their domain.

Refusal as first-class output is the part I have the least good answer for. Today the MCP server returns ranked similarity, which means the trust boundary sits on the consumer agent: it has to look at the scores and decide whether to back off. In practice, an LLM in the agent loop will happily turn a 0.62 cosine into a confident sentence. The shape of the fix is roughly:

1. Server-side threshold gating. Below a per-relationship similarity floor, return an explicit no_match / empty result rather than a top-K with low scores. The trust boundary moves from agent to server.
2. Tiered confidence in the response itself (high / low / none) rather than a single score the agent re-interprets. Removes the "is 0.7 good?" judgment from the consumer.
3. Pair with the owner-sampling check so the thresholds stay calibrated. Otherwise the threshold becomes another opaque number we can't defend.

The shape I keep landing on is that absence and uncertainty are different failure modes and want different signals: absence is just honest, uncertainty at least lets the consumer ask a follow-up, but plausible-but-wrong is invisible to a non-engineering consumer. That last one is where the server has to prevent the bad answer at source rather than rely on the agent to decline.

Appreciate the read; the threads have sharpened the framing more than the original article did.

[Nazar Boyko]

On the no dynamic analysis gap at the end, one thing that might fold in cleanly is treating production call counts as weights on the edges you already have. You're not adding a new graph, just annotating the existing SAME_ENTITY and call edges with how often each one actually fired last week, and the dead edges fall out as the ones that never lit up. It also gives your blast radius queries a way to sort by real traffic instead of treating every reachable path as equally likely. The choice to annotate only the boundaries is the part I found most convincing, since it's the same instinct as documenting a system for a new teammate. You explain the seams, not every private helper.

[Ryosuke Tsuji]

Yes, that's exactly the direction I want to go. Annotating existing static edges with runtime weights (call counts, last-fired timestamps) rather than building a parallel graph fits naturally on top of the boundary-level structure we already have.

At the boundary granularity (API / Event / Page entrypoints), this works cleanly: how often a boundary fires last week is a reasonable importance and freshness signal. The complication shows up when you push the weighting inside the boundary and start annotating individual call edges / nodes along the internal path. There, what the count means depends on whether the path is happy or error. On a happy path, the usual logic holds (frequent = important, rare = candidate for cleanup). For error-path code (catch blocks, retry handlers, fallback branches), frequent execution still carries value because it surfaces upstream problems worth investigating. But infrequent execution doesn't imply the handler is unimportant. A rarely-fired catch block might be the safety net that's quietly kept the system intact all year.

So for internal-path weighting, you'd want to classify the execution path itself first (happy / error / recovery / one-off init) before you can
interpret what the count actually means. I don't have a clean answer for how to do that classification reliably yet. Some combination of structural cues (try/catch position, branch reachability) plus runtime correlation with error logs / 5xx rates seems plausible, but it's the main open problem for me on the dynamic side.

[Kartik N V J K]

Embedding raw function bodies tends to bury the signal under boilerplate, so fixing the entry-point problem first makes sense. Did you embed a generated summary of each node instead of the source span, and if so how do you keep those summaries from drifting as the code changes under them? Static-analysis-first seems like the only way the graph stays trustworthy across 46 repos.

[Ryosuke Tsuji]

Yeah, that's basically the split. One AI writes annotations on each PR diff,
another reviews that they still match the code.

A few extra details worth flagging:

1. We don't embed a runtime-generated summary, we embed the @graph-business JSDoc field that the annotation AI commits directly above each declaration.
   It's 1-2 sentences of business semantics co-located with the code, so the thing being indexed is small, human-readable, and survives in git history.

2. Drift detection is mostly free: since the annotation lives in JSDoc on top of the declaration, any code change in that file pulls the annotation into the same PR diff. The reviewer AI sees both sides side-by-side.

3. Both AIs have a hard rule that they can ONLY modify JSDoc, never
   application logic. The reviewer enforces it (any non-JSDoc diff line is
   treated as REQUEST_CHANGES). Without this constraint the two roles could cooperatively rewrite code "to keep things consistent", which is exactly the drift you'd want to prevent.

4. Static analysis (tree-sitter) still owns the structural part of the graph (files, functions, imports, calls). The AI layer only fills business-level metadata (@graph-business, @graph-connects), so the drift surface is the annotation layer only, never the structural backbone.

And yes, static-analysis-first is exactly why this holds across 46 repos. The annotation layer just adds the part static analysis can't see
(e.g. "this endpoint is the Slack OAuth callback"), and the two AI roles
keep that layer in sync with the code.

[Mudassir Khan]

the annotation maintenance loop has one structural risk: the AI that writes and the AI that reviews likely share model weights. a systematic generation error might look plausible to a reviewer with the same priors. the SLO catches missing coverage but not plausibly wrong coverage. the owner sampling Mike proposed closes that gap. what I am curious about is the annotation generation trigger: when a new boundary appears in a diff, the webhook fires and the AI annotates what is in the diff. but if a new route is registered in a config file and the actual handler lives in a file not touched by that diff, does the pipeline catch the handler as needing annotation, or does it slip through and stay unannotated until the SLO eventually fires?

[Ryosuke Tsuji]

Great decomposition on both sides.

On shared model weights, you're right, and Mike's owner sampling is exactly what closes that gap. AI review reduces the review load; it doesn't eliminate it. A small sample of ground-truth eyes stays in the loop specifically for the "plausibly wrong but covered" failure mode the SLO can't see.

On the config-in-diff-but-handler-elsewhere case, in principle it should catch it, and this is where the code-graph approach diverges most clearly from diff-only SaaS AI Reviewer bots. Annotation-author and annotation-reviewer both traverse via the code graph, not the diff alone, so they can reason about code that never appears in the PR diff at all. When a new route registration lands in a config file, the graph resolves the handler that route points to (the boundary → handler edge that static analysis extracts up front). The pipeline sees "this route is new, and its handler in some-other-file.ts has no @graph-* annotation" as one connected piece of work, even when the handler file isn't in the touched set. That gets queued for annotation on the same PR.

Where this would actually slip is if the config → handler resolution itself is dynamic in a way static analysis can't follow. Handler paths constructed from runtime variables, dynamic imports from computed strings, and similar patterns fall back to the SLO picking them up later. That's honestly the weakest link in that chain.