Why AI coding agents struggle with Figma (and what actually worked)

echo-ae — Thu, 30 Apr 2026 21:21:59 +0000

AI coding agents have become surprisingly capable at generating frontend code. In isolated tasks they often perform well, but once you try to plug them into a real design workflow — especially one based on Figma — the results tend to degrade.

What I kept running into wasn’t really a limitation of the models themselves. It was the way design data is passed into them.

A common instinct is to give the agent as much context as possible: the entire Figma file, or at least a large portion of it. On paper that sounds reasonable. In practice, it makes things worse.

Figma files contain a lot more than just the structure of the UI. They include deeply nested trees, inconsistent naming, layout artifacts, and a significant amount of visual metadata that isn’t particularly meaningful for code generation. What looks like a clean design to a human turns into a fairly noisy representation when you inspect the underlying data.

Once that data is fed into a model, the signal-to-noise ratio drops quickly. The agent starts making weaker assumptions about hierarchy and intent, and the output becomes less predictable. You get code that is technically plausible, but often needs a lot of correction.

I also tried going through the usual paths — using Dev Mode or APIs to extract design information. While that helps with access, it doesn’t really solve the core issue. You still end up working with raw data that wasn’t designed for machine interpretation, and you introduce additional dependencies along the way.

What ended up working better for me was not adding more data, but reducing and reshaping it.

Instead of treating the design file as something to pass through directly, I started thinking of it as something that needs to be interpreted first. The idea was to extract only the part of the design that actually matters for the task at hand, clean it up, and present it in a way that is easier for an agent to reason about.

Here’s what the workflow looks like in practice:

Select → export → normalize → agent generates UI

A local pipeline

That led me to build a small local pipeline.

On the Figma side, I export a selected fragment of the design — usually a frame or a component — without relying on Dev Mode. That data is then processed locally, where the structure is simplified and normalized. The goal isn’t to perfectly preserve every detail, but to make the hierarchy and relationships between elements clearer.

The processed result is exposed through a local MCP server, which allows an AI agent to query it as structured context rather than consume raw JSON. At that point, the model is no longer trying to interpret Figma directly. It’s working with something that is closer to a logical representation of a UI.

Why this helps

An interesting side effect of this approach is that it naturally limits the scope of what the agent sees. Not by manually trimming input, but by only providing a clean, relevant slice of the design. That alone makes a noticeable difference.

In practice, the outputs became more consistent. The model made fewer strange layout decisions, and the generated code required less manual fixing. The overall workflow felt less like “prompting and hoping” and more like something you can rely on.

Local-first advantages

Keeping everything local turned out to be another advantage. There’s no dependency on external APIs, no usage-based cost, and no requirement to enable specific Figma features. It also makes the setup easier to control and reason about.

Open source

I ended up open-sourcing the tool I built around this approach:

https://github.com/echo-ae/local_figma_port

The implementation itself is still evolving, but the underlying idea has been surprisingly stable: giving the model less, but better-structured context leads to better results.

Discussion