Building Diagram AI: From Natural Language (and Screenshots) to AWS Architecture Diagrams

Drawing architecture diagrams by hand is tedious. You drag boxes around, align icons, fix arrows, and then your design changes and you do it all again. I wanted something where I could just describe an architecture in plain English — or paste an existing diagram — and get a clean, AWS-style diagram back.

This post is the story of building Diagram AI: a browser app that turns natural language and images into architecture diagrams. It runs on Vercel, uses Amazon Bedrock (Claude Haiku 4.5) for the language understanding, and renders the actual diagrams with AWS's open-source diagram-as-code tool. I'll be honest about the parts that didn't go smoothly, because that's where the real lessons were.

The core idea

The pipeline is simple to describe:

[User input: text or a diagram screenshot]
        │
        ▼
[Vercel · Next.js]
   ├─ /api/generate-yaml  → Bedrock Claude Haiku 4.5 → diagram-as-code YAML
   └─ /api/render         → AWS Lambda (Function URL) → awsdac → PNG
        │
        ▼
[Browser: live preview + PNG download]

The clever part is the middle layer. Instead of asking an LLM to "draw a diagram" (which it can't do reliably), I ask it to write YAML in the diagram-as-code schema. A deterministic CLI then renders that YAML into a real, standards-compliant AWS diagram. The LLM does the understanding; a proven tool does the drawing.

Why diagram-as-code?

diagram-as-code (the awsdac CLI) is an AWS Labs project that generates architecture diagrams from human-readable YAML. It ships official AWS service icons and follows AWS diagram conventions. Crucially, it's extensible: you can add your own definition files to draw non-AWS services too. That extensibility became important later.

First wall: you can't just import it

My initial plan was elegant on paper: diagram-as-code is written in Go, so I'd compile it into a Vercel Go Serverless Function and call it as a library. Clean, all-in-one, no extra infrastructure.

Then I read the source. Almost all of the core logic lives under Go's internal/ directory — internal/ctl, internal/types, internal/definition, and so on. Go's language rules forbid importing internal/ packages from outside the module. The library-import approach was simply impossible.

So I pivoted: run awsdac as a CLI inside AWS Lambda, exposed via a Function URL, and have the Vercel app call it over HTTPS. Vercel handles the frontend and the LLM; Lambda handles the rendering.

Second wall: no Docker, no Go on the machine

The plan was to package the Lambda as a container image. But the build machine had neither Docker nor Go installed. Rather than fight that, I switched to a zip-based Lambda on the Python 3.12 runtime:

• Download the official prebuilt awsdac Linux binary from the GitHub releases.
• Write a tiny Python handler that shells out to the binary.
• Zip the two together and upload. No Docker, no Go build step.

The handler is small — receive YAML, write it to /tmp, run awsdac, return the PNG as base64:

proc = subprocess.run(
    [AWSDAC_PATH, "--allow-untrusted-definitions", in_path, "-o", out_path],
    capture_output=True, text=True, env={**os.environ, "HOME": "/tmp"}, timeout=25,
)

Third wall: a 403 that wasn't in any policy

With the function deployed, direct aws lambda invoke worked perfectly — a clean PNG came back. But calling the Function URL over HTTPS returned 403 Forbidden / AccessDeniedException.

The auth type was NONE. The resource policy granted lambda:InvokeFunctionUrl to *. Everything looked right. I checked for org-level SCPs and RCPs — none.

The answer was in a documentation note: since October 2025, public function URLs require two permissions — both lambda:InvokeFunctionUrl and lambda:InvokeFunction. I only had the first. Adding the second statement fixed it instantly:

aws lambda add-permission --function-name diagram-ai-render \
  --statement-id FunctionURLAllowInvoke \
  --action lambda:InvokeFunction --principal "*" \
  --invoked-via-function-url --region us-east-1

Lesson: when a 403 makes no sense, check whether the platform's rules changed recently, not just your own config.

Making the LLM output good diagrams

Getting valid YAML was the easy half. Getting good-looking diagrams took several rounds of prompt engineering, each driven by an ugly output:

1. Arrows piercing containers. The LLM would draw an arrow from User straight to an ALB deep inside a VPC, slicing through the "AWS Cloud" and "VPC" boxes. Fix: a hard rule that any external→internal arrow must route through an Internet Gateway placed on the VPC's border (BorderChildren), exactly like AWS's own reference diagrams.

2. Arrows overlapping label text. diagram-as-code pins each label directly under its icon and routes arrows through the icon's center, so vertical flows always crossed the text. After testing four layout strategies, the winner was horizontal flow (Direction: horizontal): horizontal arrows pass through the icon's mid-height and clear the labels below.

3. Invented resource types. Asked for a "box," the model confidently produced AWS::Diagram::Container — a type that doesn't exist. I added an explicit list of safe AWS types plus a rule: never invent types; fall back to a generic resource if unsure.

Each fix went straight into the system prompt with a concrete good/bad example. The prompt grew long, but the outputs got dramatically cleaner.

Going beyond AWS

diagram-as-code is AWS-first, but its definition-file mechanism lets you add arbitrary icons. I wanted hybrid diagrams — "GitHub → Vercel → AWS Lambda" — to look right.

I built a small pipeline: pull SVGs from simple-icons, tint them with each brand's color, and convert to 128×128 PNGs with sharp. That produced 19 external service icons (Vercel, Netlify, Cloudflare, GitHub, Supabase, Auth0, OpenAI, Anthropic, Stripe, and more), bundled into an icons.zip plus an external-icons.yaml definition file that maps types like External::Vercel to the icons.

One subtlety: awsdac refuses to load definition files from URLs outside the official repo unless you pass --allow-untrusted-definitions. And I originally hosted the icons on GitHub Releases, but a private-repo mix-up left GitHub's CDN serving cached 404s. The robust fix was to serve the definition file and icon zip from Vercel's own /public folder — same origin as the app, instantly updatable on each deploy.

I also added Type: Group definitions so you can draw a branded container — e.g. a "Vercel Platform" box with the Vercel logo in the corner, holding child resources inside.

Reading existing diagrams

The feature I'm happiest with: upload a screenshot of an existing diagram and regenerate it. Claude Haiku 4.5 on Bedrock is multimodal, so I extended the API to accept an image (base64) alongside or instead of text.

Three modes fall out of this naturally:

• Image only → read the diagram and reproduce an equivalent YAML.
• Image + text → transform it: "take this diagram but swap Cloudflare for Vercel and EC2 for ECS."
• Text only → the original behavior.

One build-time gotcha worth noting: the Bedrock client was being constructed at module load. During next build's page-data collection (which runs without AWS env vars) that threw Region is missing. Moving the client into a lazy, request-time factory fixed it — and it's a good pattern regardless.

The frontend supports paste (⌘V), drag-and-drop, and file selection, with a 5 MB cap.

Shipping it: CI/CD

Finally, I wired up automatic deploys. Connecting Vercel's GitHub App via CLI didn't work (it needs a browser step for repo access), so I used GitHub Actions + the Vercel CLI instead: on every push to main, the workflow runs vercel pull / build / deploy --prebuilt --prod with a token stored in GitHub Secrets.

The one hiccup: the Vercel token expired mid-way, producing The token provided via --token argument is not valid. Regenerating it (with no expiry) and re-registering the secret got the pipeline green. Now git push is all it takes to ship.

What I'd tell my past self

• Read the source before committing to an integration strategy. The internal/ package issue would have saved a day if I'd caught it on day one.
• When auth fails inexplicably, suspect recent platform changes. The dual-permission Lambda URL requirement was invisible in my own config.
• For LLM-generated structured output, encode taste as rules with examples. "Don't pierce labels" only worked once it became a concrete layout rule with a good/bad sample.
• Same-origin hosting beats clever CDN tricks when you control the deploy anyway.

The stack, in one line

Next.js on Vercel · Amazon Bedrock (Claude Haiku 4.5, text + vision) · AWS Lambda (zip, Python 3.12) running the awsdac CLI · simple-icons + sharp for non-AWS icons · GitHub Actions for CI/CD.