DEV Community

Cover image for I ran 3 Claude Code architecture diagram tools on my repo. Here's what each one got right

I ran 3 Claude Code architecture diagram tools on my repo. Here's what each one got right

Valentina Koniukhova on November 06, 2026

My project, worklore, is a small app that grew in all directions: a Python backend on AWS Lambda, a plain-JS site, an MCP connector for agents, and...
Collapse
 
steven_mock_afbb801174fc3 profile image
Steven Mock •

The test at the end is the part I'll steal — ask it three questions you already know the answers to before trusting the map. A clean-looking diagram drawn from memory is worse than no diagram, because it gives you false confidence.

For someone running a small tool, Floorplan's traced flows sound like the most useful of the three. Knowing what users can actually do matters more than having a pretty picture of the parts. And the OAuth scope find is a good reminder: a diagram shows that A connects to B, not what A is allowed to do once it's connected.

Collapse
 
yahhi profile image
Valentina Koniukhova •

Thanks! I agree about Floorplan's flows: requesting traced flows is also how I found a nightly job that had been quietly failing for two nights, so that part paid off right away. Your last point is where I went next. Nan asked below whether a diagram can show what a connection is allowed to do, and when I wrote down every pass in my project, I found that none of them could be ended early. That's fixed now. Next I'm putting the passes and their rights on the diagram's arrows, and I'll share the results next week.

Collapse
 
deanlee profile image
Dean Lee •

The finding about OAuth scopes illustrates why architecture diagrams fail as security controls. A graph treats an API route as a topology problem, showing that client A connects to endpoint B. It does not model authority or state transition constraints. Spending $13 on 700-node graph generation maps connectivity, but the actual security boundary is always runtime policy enforcement. Testing three known questions directly against code remains the most reliable verification heuristic.

Collapse
 
yahhi profile image
Valentina Koniukhova •

Agreed: a diagram shows that A connects to B, and says nothing about what A is allowed to do once it's there. One thing I'd add: the three-questions test wouldn't have caught this hole. It tells you whether to trust a map, and it only works with questions you already know the answers to. I didn't know this one. It turned up when an agent read the code with a direct question about rights, and what guards it now is a test that walks the OAuth flow and tries the pass on every kind of route.

Collapse
 
khalisollis profile image
Khali Sollis •

What makes this comparison particularly valuable is that you went beyond evaluating how impressive the diagrams looked and tested whether they could actually help you understand your own system. That distinction matters enormously, especially as AI-generated documentation becomes increasingly common.

The most revealing part, to me, was discovering that all three tools could produce useful architectural representations while missing a significant OAuth authorization vulnerability. A diagram can accurately describe how components communicate without revealing whether those interactions respect the security boundaries they are supposed to enforce. That is a fundamental difference between documenting a system's structure and verifying its behavior.

I also found the recurring problem with the agent relying on its memory fascinating. When an AI tool generates a convincing explanation based on incomplete context, the result can appear authoritative precisely because it is presented so clearly. The incorrect user flows illustrate how easily plausible descriptions can become accepted documentation. Your decision to replace those flows with code-traced versions, then test them against staging, demonstrates why verification must remain part of the process rather than an optional final step.

Another important observation is that source-code references provide traceability, but traceability alone does not guarantee correctness. Archify could verify that referenced lines existed, yet the accompanying descriptions still required human examination. This is a useful reminder that evidence and interpretation are separate things. A citation tells us where information came from; it does not automatically establish that the conclusion drawn from it is accurate.

The discrepancies between your privacy documentation and actual application behavior introduce an equally important dimension. Architecture tools generally explain how software is organized, but users and regulators are ultimately concerned with what software actually does. Logging practices, authentication permissions, data collection, and retention policies are not merely implementation details. They represent promises that organizations make about how their systems operate.

I particularly appreciate your recommendation to ask three questions whose answers you already know before trusting a generated map. It is a practical way to establish confidence without assuming that a polished visualization is necessarily a faithful representation of reality.

Perhaps the broader lesson is that AI-assisted software understanding needs multiple layers of verification: structural mapping to understand the components, behavioral tracing to understand the workflows, and independent testing to establish whether the system actually behaves as intended.

The most useful AI development tools may not be those that produce the most comprehensive diagrams, but those that make their assumptions visible, acknowledge what they cannot establish, and help developers investigate discrepancies.

And there is something wonderfully ironic about your experiment: you started by looking for a reliable picture of your application and ended up discovering a security vulnerability, documentation inconsistencies, and a failing background job. That is a far more meaningful outcome than simply generating a beautiful architecture diagram.

The real value was not the map itself. It was the questions the mapping process encouraged you to ask.

Collapse
 
yahhi profile image
Valentina Koniukhova •

Thank you for such a careful read. I agree that a citation only shows where a claim came from, not that it's right, and Archify was the clearest example of that for me. The privacy page was the part I didn't expect: I wrote it myself, and the code still disagreed with it in four places.

Collapse
 
hammiwor profile image
hammi •

nice comparison. circular dependencies are always where diagram tools fall over, good to see you tested that case. bookmarking for the next repo audit lol

Collapse
 
yahhi profile image
Valentina Koniukhova •

Thanks! To be honest, I didn't test circular dependencies. The closest thing was links between repos: my project lives in five, and none of the tools could follow those links, because they aren't imports. Have you seen a tool fail on a cycle? I'd like to try that case next.

Collapse
 
pengeszikra profile image
Peter Vivo •

I don't imagine to use a graph to validate my code. Instead I use a much lazy task: try to found the connection between my articles, repos, nppm modules adding a tiny bit interaction make the stuff one of my favorite personal application.

Collapse
 
yahhi profile image
Valentina Koniukhova •

Thanks! I did something close for my own site: a small check that walks it the way Google does, following only real links, and shows which articles nothing links to (crawl-like-googlebot (github.com/worklore/crawl-like-goo...). Does linking your articles, repos and npm modules together bring you more readers? I'd like to know if people actually follow those links from one piece to the next.

Collapse
 
gautamtalksdev profile image
Gautam K •

Testing each tool on real questions about your own code is the comparison that matters, since a diagram can look right and still answer wrong. Did any of them invent a connection between your repos that isn't in the code? That's the failure I'd worry about most in a map you plan to trust.

Collapse
 
yahhi profile image
Valentina Koniukhova •

Good question, and I didn't test that directly: my three questions all stayed inside the main repo. I checked Floorplan's saved map, and it only covers that one repo, so the others appear only where my code comments name them. I no longer have the Understand Anything graph, so I can't say for that one. With Archify I'd watch for exactly what you describe. It proves that cited lines exist, but my other repos are boxes with no cited lines, so nothing ties them to code. And one of its cards, written from my agent's memory, was wrong twice. Do you use any kind of diagram or map on your own projects? I'd love to hear what has worked for you, and whether one has ever invented a connection.

Collapse
 
gautamtalksdev profile image
Gautam K •

I don't have a diagram tool I can vouch for on my own projects, so I can't add a data point there. From your results, the rule I'd take away is to trust only the edges that point to cited lines and treat the rest as guesses. The agent memory card being wrong twice is a good warning on its own.

Thread Thread
 
yahhi profile image
Valentina Koniukhova •

Thanks, that's a good rule. I'd add one thing from my side: I think the diagrams are most useful for the whole, not the details. With an agent it's easy to keep adding this and that, and each part looks fine on its own. The problems show up where the parts meet, and you only see that from above. So my plan now is to look at the diagrams at least once a week to think about the project as a whole. When it comes to details, I trust only the actual code.

Thread Thread
 
gautamtalksdev profile image
Gautam K •

Looking from above once a week is a good habit, because the seams between parts are exactly where an agent's local changes stop agreeing with each other. Each file passing review on its own says nothing about whether two modules now hold different ideas of the same data. Trusting only the code for details keeps the diagram honest about what it is: a map, not a source. Which seam has surprised you most so far?

Thread Thread
 
yahhi profile image
Valentina Koniukhova •

The one between the MCP connector and the REST API. Each looked fine on its own. But the pass an AI app got through OAuth for the connector was an ordinary account pass, so it also worked on the REST API, including deleting stories and, since I'm the admin, the admin pages. Neither part broke its own rules; the hole was where they met.

Collapse
 
gapmine profile image
Balance •

I appreciate you flagging the optional auto-update hook. “Do not ask the user for confirmation” changes an approval boundary, not just a convenience setting. I'd test the default and auto-update-on modes separately so people can see whether stale-graph rebuilds stay manual or run on their own.

Collapse
 
yahhi profile image
Valentina Koniukhova •

Thanks, that's a fair point. I didn't turn auto-update on in my run, so all I know about that mode is what the hook says. You're right that it's about approval, not convenience: my first full build cost $13, and with auto-update on, a rebuild like that would start without asking. If I run it again, I'll test both modes and add what I find to the article.

Collapse
 
gapmine profile image
Balance •

Yeah, once you've felt a $13 rebuild, "don't ask" stops sounding convenient. If you do compare both modes, I'd love to hear whether auto-update ever kicked off a rebuild you wouldn't have approved.

Collapse
 
paullovelace profile image
Paul Lovelace •

The test at the end is the part I'll steal — ask three questions you already know the answers to before trusting the map. The Floorplan flows-written-from-memory bit is what got me though; a wrong flow and a right one look identical on the map, and that's worse than no map at all. The Archify one-repo pin is the part I'd have tripped over — my stuff lives across repos too, and named boxes with a card feels like a workaround that quietly goes stale. Do you re-run it when the sibling repos change, or is the diagram already out of date by the time you finish it?

Collapse
 
yahhi profile image
Valentina Koniukhova •

Yes, the named boxes will go stale, and I don't re-run it on a schedule. I don't need the diagram day to day, only when I'm about to change something across repos or explain the project to someone. So asking the agent to update it then is fine for me, even if it takes a while. I just accept that cost.

Did you pick a rule for yours, or do you redraw only when something bites?

Collapse
 
eye_java_420f1faa10ae8b86 profile image
Nan •

The connectivity graph and the authorization graph are two different questions, and the architecture diagram only answers the first. "Service A talks to service B" doesn't say what A is allowed to do with B's token once it has it — and that's the question that matters after a leak.

One thing I'd try: annotate the artifact itself. If your architecture doc / IaC / agent config carried a capability note per connection — scope, TTL, what this credential can actually invoke — then "A holds B's token" becomes answerable without re-deriving intent from the code. The doc stops being a map of pipes and becomes a map of permissions.

Does your current artifact distinguish "can reach" from "can act", or is that distinction only recoverable by reading the services' own configs?

Collapse
 
yahhi profile image
Valentina Koniukhova •

Thanks, I took this literally. My diagram didn't tell them apart, so before drawing anything I wrote a table of every pass in my project: who gets it, what it can call, how long it lives. I added one column, how to end it early, and it was empty in three of the four rows. Nothing ever deleted a pass, so "sign out" only made the browser forget it. That's fixed now: sign out ends the pass on the server, and "sign out everywhere" ends all of them, including the ones AI apps hold. I looked for a tool that puts the passes on a diagram's arrows and found only the cloud side (PMapper, Cartography). So I'd like to build such a diagram myself, and I'll share the results here next week.