Pair agent UI work with visual regression before you trust green CI

Unit tests pass. Typecheck is green. Playwright happy-path clicks through. The agent PR still ships a button that jumped 8px, a modal that lost its shadow, and a table whose sticky header now overlaps the first row.

I stopped treating “green CI” as proof that agent UI work is safe. Green means the logic didn’t break. Layout and visual regressions need their own gate.

Why agents are especially good at silent UI breaks

Agents rewrite JSX and CSS with confidence. They:

• Swap gap for margin “to match the design”
• Drop a wrapper that “wasn’t doing anything” (it was)
• Change flex → grid mid-component without updating children
• Tweak token usage that only shows up at a specific viewport
• “Clean up” class names and lose a one-off override that fixed Safari

None of that fails a unit test. Snapshot DOM tests often miss spacing. E2E that asserts text still passes. Humans catch it in review — if they open the preview and squint hard enough.

What I mean by visual regression here

Not a full design QA suite on day one. A thin, blocking check on the surfaces the agent is allowed to touch:

1. Capture baseline screenshots of key states (default / hover / error / empty / mobile)
2. On the PR, re-capture the same states
3. Fail (or require human ACK) when pixels drift past a small threshold

Tools I’ve used in this role: Playwright screenshot diffs, Chromatic / Lost Pixel / Argos on Storybook stories, or a cheap custom Playwright compare in CI. The brand matters less than blocking merge when the diff is real.

Pair it with the agent prompt — don’t bolt it on after

If visual checks only run after the model already rewrote half the page, you burn a review cycle. I put the expectation in the task:

## Visual regression (HARD for UI diffs) You are editing UI. Before you finish:  1. List the stories / routes / states you changed (default, hover, focus, error, empty, sm/md breakpoints if layout changed). 2. Do not claim “looks fine” without a screenshot or story URL. 3. Prefer smallest CSS/DOM change that keeps existing visual baselines. 4. If you must change layout structure, say so explicitly and list which baselines will need an intentional update. Forbidden: drive-by spacing “cleanup”, deleting wrappers without checking computed layout, renaming classes that might be used in CSS modules / global CSS. 

The agent still ships UI. It just knows silent restyles are not free.

Minimal Playwright gate that actually catches agent PRs

I don’t need full Storybook coverage on every legacy page. For agent-heavy areas I pin a few routes:

// e2e/visual/agent-surfaces.spec.ts import { test, expect } from '@playwright/test'; const surfaces = [ { name: 'settings-billing', path: '/settings/billing' }, { name: 'checkout-summary', path: '/checkout/summary' }, { name: 'empty-projects', path: '/projects?empty=1' }, ]; for (const s of surfaces) { test(`visual: ${s.name}`, async ({ page }) => { await page.goto(s.path); await expect(page).toHaveScreenshot(`${s.name}.png`, { maxDiffPixelRatio: 0.01, animations: 'disabled', }); }); } 

maxDiffPixelRatio: 0.01 is strict enough that an 8px shift fails, loose enough that font hinting noise rarely pages me. Tune per app.

When an agent PR intentionally changes the look, update the baseline in the same PR with a one-line note: VISUAL_OK: billing card padding 16→20 per design. Silent baseline refreshes are how regressions hide.

Storybook is even better when you already have stories

If the component has stories, prefer visual diffs on stories over full-page shots:

• Faster
• Stable (less app chrome / data noise)
• Matches how agents should work anyway — “edit Button story states, not random pages”

My rule for agent UI tasks: no story, no merge for shared components. Either the agent adds a story for the state it touched, or a human already had one. Page-level screenshots are the fallback for app shells.

What green CI means after this

Agent PRs that are “all green” without a visual gate are only green on the first two rows. That’s incomplete.

Review comments I actually leave

When I see a UI-only agent diff with no visual evidence:

Visual gate missing: this touches checkout summary layout. Add/update Playwright screenshot (or Storybook + Chromatic) for default + error + mobile, or paste preview screenshots in the PR. Don’t merge on typecheck alone.

When baselines change without explanation:

Baseline bump needs VISUAL_OK: <why>. Pixel drift without a design reason is a regression, not a cleanup.

What I stopped trusting

• “I checked the preview” with no artifact
• DOM snapshots as a substitute for pixels
• Full E2E coverage as proof of layout
• Agent claims that spacing changes are “equivalent”

Pair agent UI work with visual regression before you trust green CI. Keep the suite small, block on drift, and make intentional visual changes explicit. The unit tests can stay green — your users still see the 8px jump.