I built an app where you type a sentence and a real, editable design appears on a Penpot canvas

Every AI UI tool ends the same way. You get a picture, or you get a pile of React you
did not ask for. Neither is a design. A picture cannot be opened by a designer. A
generated component is a diff nobody reviewed.

I wanted the middle option: an agent that produces an actual design artifact, in a
tool a human can keep working in. So I built Atelier, a small local app where you
describe a screen, and a real, editable board appears on a Penpot canvas.

Type this:

A pricing page hero for a developer tool, dark theme, three tiers

Get a board in your Penpot file, plus the JavaScript that built it, plus a PNG.

Why Penpot

Penpot made this possible in a way Figma cannot right now, for three reasons.

First, it is open source and self-hostable, so I can run the whole stack locally and
nothing leaves my machine.

Second, since 2.18 it ships an MCP server in its own docker-compose stack. The
official penpot-mcp container is already there when you stand up Penpot. No separate
install.

Third, and this is the interesting part, Penpot's canvas is not a proprietary
format. It is real web structures with a real Plugin API. So an agent writing Plugin
API JavaScript is writing to the same primitives a designer manipulates by hand. The
output is genuinely editable.

That last point is what makes it a design tool rather than a renderer.

The architecture

Atelier is a React app plus a small local API. The API is mounted as a Vite plugin, so
there is no second process to run and no proxy config to maintain.

Three visible steps: the model writes code, the code runs on the canvas, the board is
exported back.

Why the API is server-side

Two reasons, and the first one cost me an afternoon.

CORS. Neither the Penpot MCP endpoint nor most model endpoints send CORS headers.
A browser fetch straight at them is blocked. Server-side, no such rule exists.

Session handling. MCP over Streamable HTTP gives you an mcp-session-id response
header on initialize that you must echo on every later call. Doing that in the browser
means holding protocol state in React. Doing it in Node is three lines.

The plugin mounts into both dev and preview, so one command serves the UI and the API:

function atelierApi(): Plugin { return { name: 'atelier-api', configureServer(server) { server.middlewares.use(apiMiddleware); }, configurePreviewServer(server) { server.middlewares.use(apiMiddleware); }, }; } 

A dependency-free MCP client

The MCP spec looks heavier than it is. Over Streamable HTTP the whole thing is one POST
shape plus a session header, so I wrote the client by hand in about 150 lines and added
zero dependencies.

const res = await fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json', Accept: 'application/json, text/event-stream', ...(sessionId ? { 'mcp-session-id': sessionId } : {}), }, body: JSON.stringify(body), signal: controller.signal, }); 

Two things I did not expect.

Penpot replies as SSE, not JSON. The response body is event: message followed by
data: {...}. So you parse frames, not a document.

for (const rawLine of text.split(/\r?\n/)) { const line = rawLine.trimStart(); if (!line.startsWith('data:')) continue; sawSse = true; const payload = line.slice(5).trim(); if (!payload || payload === '[DONE]') continue; try { messages.push(JSON.parse(payload)); } catch { /* partial frame */ } } 

execute_code returns its payload as a JSON string, not structured data. You get a
text block whose contents are {"result": ..., "log": ""}. If you do not unwrap that,
every downstream check silently reads undefined.

export function unwrapExecuteResult(text: string): unknown { try { const parsed = JSON.parse(text) as Record<string, unknown>; if (parsed && typeof parsed === 'object' && 'result' in parsed) return parsed.result; return parsed; } catch { return text; } } 

The traps, and how I found them

This is the part worth reading, because these are the things that make a naive version
of this app look broken while reporting success.

Trap 1: text renders one character per line

My first generated screen came back with a perfectly reasonable summary object. Three
children, correct names, correct types. It looked like it worked.

It had not worked. The headline was a vertical stack of single letters.

A freshly created Penpot Text shape is 1px wide. If you leave it alone, the browser
wraps it one character per line. The returned data is fine. The pixels are ruined.

I caught it by not trusting my eyes. I exported the board, then measured the bounding
box of the text colour in the PNG:

Two numbers made it unambiguous. The fix is to give text an explicit width and then let
the height follow:

const t = penpot.createText('Headline'); t.resize(1200, 20); // explicit width t.growType = 'auto-height'; // height follows the wrapped content t.align = 'center'; t.fontSize = 72; 

Note the trap inside the trap: do not reach for growType = 'auto-width' on a shape
you just created. It starts at 1px and will not reliably reflow. Pick a width that fits
your layout.

Trap 2: exporting the page fails

The obvious export call is export_shape with the special id page. It fails, because
the page root frame is zero sized and Penpot's exporter waits for an element that never
becomes visible:

locator.waitFor: Timeout 10000ms exceeded. waiting for locator('#screenshot-00000000-...') to be visible locator resolved to hidden <svg width="0.01" height="0.01" ...> 

So the export endpoint resolves the board by name first, then exports it by id.

Trap 3: the endpoint needs a token, and static calls hide it

The official Penpot MCP container always runs with --multi-user. Every session must
carry a per-user token:

Error: No userToken found in session context. Multi-user mode requires authentication. 

Here is the nasty part. high_level_overview returned a full, cheerful answer with no
token and no plugin connected, because it is static text. It looks exactly like a
working integration. Only a call that touches the file reveals the truth.

That taught me a rule I now assert in the test suite: never use a static tool as a
health check.

The rest of the list

• Hex colours must be uppercase.
• fills and strokes are replaced, never mutated. Assign a whole new array.
• Add flex layout before appending children, or the existing order gets scrambled.
• width and height are read-only. Use resize().
• Re-running must be idempotent, so the generated code deletes any existing board named Screen first. Otherwise boards pile up on the canvas.

All of these now live in the system prompt, so the model avoids them automatically
instead of me fixing them per request.

Verification without a live Penpot

The awkward problem: how do you test an integration whose other half is a design tool
you have to run?

I wrote a mock MCP server that deliberately reproduces Penpot's real behaviour. SSE
framing, the session header, the token requirement, the "no plugin connected" error,
execute_code returning a JSON string, export_shape returning a base64 image block.

Then the suite drives the real middleware and the real MCP client against it and
asserts 30 behaviours across ten groups. The one I like most does not trust the string
prefix at all. It decodes the base64 and checks the magic bytes:

const bytes = Buffer.from(b64, 'base64'); const isPng = bytes.length > 8 && bytes[0] === 0x89 && bytes[1] === 0x50 && bytes[2] === 0x4e && bytes[3] === 0x47; 

Real results:

npm run verify -> 30 passed, 0 failed npm run typecheck -> 0 errors npm run build -> 154.7 kB JS, 50.2 kB gzipped npm audit -> 0 vulnerabilities 

The browser flow is covered too. A Playwright script drives the actual React app,
fills the real settings inputs, types a prompt, clicks Generate, and asserts the canvas
holds a PNG data URL with all three steps green and a clean console.

I also swapped Vite 5 for Vite 7 during this, because Vite 5 pulled a flagged esbuild
advisory. On a repo other people will clone, that shows up as Dependabot noise on day
one.

What I would build next

The one I want most is design tokens. Generate the screen, then create a real Penpot
token set, bind the shapes to it, and export the set as CSS variables and a Tailwind
config. That is the bridge from design to shipped code, and it is where Penpot's open
format pays off properly.

After that, an accessibility gate. The Plugin API can compute contrast ratios on
every text layer against its background. Fail the run below WCAG AA, then let the model
fix its own palette. No design tool does that automatically today.

Also on the list: a variant grid for exploring four directions at once, iterate-in-place
instead of regenerate, and a design review mode that audits an existing file for naming
drift and off-system colours.

Try it

The app is a single prompt box, three steps, and a real board at the end. It runs
locally, needs a Penpot instance and any OpenAI-compatible model, and has zero runtime
dependencies beyond React.

Everything is in the repo, including the mock server and the full test suite, so you can
verify it without a Penpot instance of your own.

git clone https://github.com/harishkotra/penpot-design-with-ai cd penpot-design-with-ai npm install npm run verify # 30 assertions, no Penpot needed npm run dev 

If you find a new Penpot trap, the most useful pull request you can send is an entry in
the traps list plus a line in the system prompt so the model avoids it too. That is how
this gets better.

Code & more: https://www.dailybuild.xyz/project/281-penpot-design-with-ai