I built an app where you type a sentence and a real, editable design appears on a Penpot canvas

Every AI UI tool ends the same way. You get a picture, or you get a pile of React you did not ask for. Neither is a design. A picture cannot be opened by a designer. A generated component is a diff nobody reviewed.

I wanted the middle option: an agent that produces an actual design artifact, in a tool a human can keep working in. So I built Atelier, a small local app where you describe a screen, and a real, editable board appears on a Penpot canvas.

Type this:

A pricing page hero for a developer tool, dark theme, three tiers

Get a board in your Penpot file, plus the JavaScript that built it, plus a PNG.

Why Penpot

Penpot made this possible in a way Figma cannot right now, for three reasons.

First, it is open source and self-hostable, so I can run the whole stack locally and nothing leaves my machine.

Second, since 2.18 it ships an MCP server in its own docker-compose stack. The official penpot-mcp container is already there when you stand up Penpot. No separate install.

Third, and this is the interesting part, Penpot's canvas is not a proprietary format. It is real web structures with a real Plugin API. So an agent writing Plugin API JavaScript is writing to the same primitives a designer manipulates by hand. The output is genuinely editable.

That last point is what makes it a design tool rather than a renderer.

The architecture

Atelier is a React app plus a small local API. The API is mounted as a Vite plugin, so
there is no second process to run and no proxy config to maintain.

Three visible steps: the model writes code, the code runs on the canvas, the board is exported back.

Why the API is server-side

Two reasons, and the first one cost me an afternoon.

CORS. Neither the Penpot MCP endpoint nor most model endpoints send CORS headers. A browser fetch straight at them is blocked. Server-side, no such rule exists.

Session handling. MCP over Streamable HTTP gives you an mcp-session-id response header on initialize that you must echo on every later call. Doing that in the browser means holding protocol state in React. Doing it in Node is three lines.

The plugin mounts into both dev and preview, so one command serves the UI and the API:

function atelierApi(): Plugin {
  return {
    name: 'atelier-api',
    configureServer(server) { server.middlewares.use(apiMiddleware); },
    configurePreviewServer(server) { server.middlewares.use(apiMiddleware); },
  };
}

A dependency-free MCP client

The MCP spec looks heavier than it is. Over Streamable HTTP the whole thing is one POST shape plus a session header, so I wrote the client by hand in about 150 lines and added zero dependencies.

const res = await fetch(url, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Accept: 'application/json, text/event-stream',
    ...(sessionId ? { 'mcp-session-id': sessionId } : {}),
  },
  body: JSON.stringify(body),
  signal: controller.signal,
});

Two things I did not expect.

Penpot replies as SSE, not JSON. The response body is event: message followed by data: {...}. So you parse frames, not a document.

for (const rawLine of text.split(/\r?\n/)) {
  const line = rawLine.trimStart();
  if (!line.startsWith('data:')) continue;
  sawSse = true;
  const payload = line.slice(5).trim();
  if (!payload || payload === '[DONE]') continue;
  try { messages.push(JSON.parse(payload)); } catch { /* partial frame */ }
}

execute_code returns its payload as a JSON string, not structured data. You get a text block whose contents are {"result": ..., "log": ""}. If you do not unwrap that, every downstream check silently reads undefined.

export function unwrapExecuteResult(text: string): unknown {
  try {
    const parsed = JSON.parse(text) as Record<string, unknown>;
    if (parsed && typeof parsed === 'object' && 'result' in parsed) return parsed.result;
    return parsed;
  } catch { return text; }
}

The traps, and how I found them

This is the part worth reading, because these are the things that make a naive version of this app look broken while reporting success.

Trap 1: text renders one character per line

My first generated screen came back with a perfectly reasonable summary object. Three children, correct names, correct types. It looked like it worked.

It had not worked. The headline was a vertical stack of single letters.

A freshly created Penpot Text shape is 1px wide. If you leave it alone, the browser wraps it one character per line. The returned data is fine. The pixels are ruined.

I caught it by not trusting my eyes. I exported the board, then measured the bounding box of the text colour in the PNG:

	ink bounding box	aspect	verdict
First attempt	194 x 894 px	0.2:1	vertical, broken
After the fix	954 x 198 px	4.8:1	horizontal, correct

Two numbers made it unambiguous. The fix is to give text an explicit width and then let the height follow:

const t = penpot.createText('Headline');
t.resize(1200, 20);          // explicit width
t.growType = 'auto-height';  // height follows the wrapped content
t.align = 'center';
t.fontSize = 72;

Note the trap inside the trap: do not reach for growType = 'auto-width' on a shape you just created. It starts at 1px and will not reliably reflow. Pick a width that fits your layout.

Trap 2: exporting the page fails

The obvious export call is export_shape with the special id page. It fails, because the page root frame is zero sized and Penpot's exporter waits for an element that never becomes visible:

locator.waitFor: Timeout 10000ms exceeded.
  waiting for locator('#screenshot-00000000-...') to be visible
  locator resolved to hidden <svg width="0.01" height="0.01" ...>

So the export endpoint resolves the board by name first, then exports it by id.

Trap 3: the endpoint needs a token, and static calls hide it

The official Penpot MCP container always runs with --multi-user. Every session must carry a per-user token:

Error: No userToken found in session context. Multi-user mode requires authentication.

Here is the nasty part. high_level_overview returned a full, cheerful answer with no token and no plugin connected, because it is static text. It looks exactly like a working integration. Only a call that touches the file reveals the truth.

That taught me a rule I now assert in the test suite: never use a static tool as a health check.

The rest of the list

• Hex colours must be uppercase.
• fills and strokes are replaced, never mutated. Assign a whole new array.
• Add flex layout before appending children, or the existing order gets scrambled.
• width and height are read-only. Use resize().
• Re-running must be idempotent, so the generated code deletes any existing board named Screen first. Otherwise boards pile up on the canvas.

All of these now live in the system prompt, so the model avoids them automatically instead of me fixing them per request.

Verification without a live Penpot

The awkward problem: how do you test an integration whose other half is a design tool you have to run?

I wrote a mock MCP server that deliberately reproduces Penpot's real behaviour. SSE framing, the session header, the token requirement, the "no plugin connected" error, execute_code returning a JSON string, export_shape returning a base64 image block.

Then the suite drives the real middleware and the real MCP client against it and asserts 30 behaviours across ten groups. The one I like most does not trust the string prefix at all. It decodes the base64 and checks the magic bytes:

const bytes = Buffer.from(b64, 'base64');
const isPng = bytes.length > 8 && bytes[0] === 0x89 && bytes[1] === 0x50
           && bytes[2] === 0x4e && bytes[3] === 0x47;

Real results:

npm run verify     ->  30 passed, 0 failed
npm run typecheck  ->  0 errors
npm run build      ->  154.7 kB JS, 50.2 kB gzipped
npm audit          ->  0 vulnerabilities

The browser flow is covered too. A Playwright script drives the actual React app, fills the real settings inputs, types a prompt, clicks Generate, and asserts the canvas holds a PNG data URL with all three steps green and a clean console.

I also swapped Vite 5 for Vite 7 during this, because Vite 5 pulled a flagged esbuild advisory. On a repo other people will clone, that shows up as Dependabot noise on day one.

What I would build next

The one I want most is design tokens. Generate the screen, then create a real Penpot token set, bind the shapes to it, and export the set as CSS variables and a Tailwind config. That is the bridge from design to shipped code, and it is where Penpot's open format pays off properly.

After that, an accessibility gate. The Plugin API can compute contrast ratios on every text layer against its background. Fail the run below WCAG AA, then let the model fix its own palette. No design tool does that automatically today.

Also on the list: a variant grid for exploring four directions at once, iterate-in-place instead of regenerate, and a design review mode that audits an existing file for naming drift and off-system colours.

Try it

The app is a single prompt box, three steps, and a real board at the end. It runs locally, needs a Penpot instance and any OpenAI-compatible model, and has zero runtime dependencies beyond React.

Everything is in the repo, including the mock server and the full test suite, so you can verify it without a Penpot instance of your own.

git clone https://github.com/harishkotra/penpot-design-ui-with-ai
cd penpot-design-with-ai
npm install
npm run verify   # 30 assertions, no Penpot needed
npm run dev

If you find a new Penpot trap, the most useful pull request you can send is an entry in the traps list plus a line in the system prompt so the model avoids it too. That is how this gets better.

Code & more: https://www.dailybuild.xyz/project/281-penpot-design-with-ai