I build a tool that turns a photo into a counted cross stitch chart: a grid where every square is one stitch in one DMC thread color. On paper it's an easy problem. Shrink the image, snap every pixel to the nearest thread, done.
That version works, and it makes charts nobody wants to stitch. Here's what I had to change, in case you ever write something similar (pixel art, mosaics, Perler beads, LEGO portraits all have the same problems).
1. Shrinking: average, don't sample
A chart is tiny. A 100 x 100 stitch pattern is already about 7 inches on 14-count fabric, so a 4000-pixel photo gets squeezed 40 times. Nearest-neighbor sampling picks one random pixel per cell and you get noise. Area-averaging every source pixel that falls inside the cell gives a much calmer starting point.
Averaging softens edges, though, and the things people care about in a pet or a face are thin dark lines: eyes, nostrils, the outline of an ear. So after the resize there's a light unsharp mask at stitch size, clamped so it never goes past the colors already around a cell. Without the clamp you get halos on flat artwork like logos.
2. Choosing the colors: k-means in Lab, then snap to real thread
DMC sells around 490 solid colors. Nobody wants to buy 150 skeins for one picture, so you need to pick maybe 20 to 40.
My first attempt was the obvious one: map every pixel to its nearest DMC color, then keep merging the least-used color into its nearest neighbor until you're under the limit. It's simple and it's wrong in a very visible way: the dog's eyes are 12 stitches out of 10,000, so they're the first thing to get merged into brown fur.
What works much better is k-means in CIE Lab space, weighted by how many stitches have each color, with each center snapped to the nearest real thread. Small but distinct areas (eyes, a red door, a sun) pull a center toward themselves and keep their own color. Two details matter:
- Seed deterministically. I start from the most common color and then repeatedly add the color farthest from the existing centers, weighted by its count (k-means++ without the randomness). Same photo and settings always give the same chart, which matters when someone comes back to buy the pattern they previewed yesterday.
- Bin before clustering. Rounding Lab values into bins keeps a 400 x 400 chart fast enough to recompute on every slider move in a Web Worker.
3. "Nearest color" needs a better distance
Plain Euclidean distance in Lab is fine most of the time and fast. It fails on very saturated colors, which sit outside the range of available thread: a vivid blue sky landed on a purple. CIEDE2000 gets those right but it's expensive.
The compromise: shortlist the 6 closest threads with plain Lab distance, then decide among those with CIEDE2000. If even the closest one is far away (the color is outside what thread can do), skip the shortlist and check every thread with CIEDE2000. Ties go to the lower index so the result stays deterministic.
4. Confetti is the real enemy
Stitchers have a name for single stitches of a different color scattered through an area: confetti. Every one means stopping, finding the thread, stitching one X and finishing it off. A photo conversion can easily have thousands, and that's the number one complaint about photo patterns.
The cleanup that made the biggest difference:
- Any stitch with no same-colored neighbor among its 8 neighbors gets the most common color around it.
- Small 4-connected patches (up to a few stitches) join the surrounding color, but only if that color is close enough (a ΔE2000 threshold). That threshold is what keeps the highlight in an eye: it's tiny but very different from what's around it, so it stays.
- Colors used for only a handful of stitches get folded into the nearest remaining color, again only within a distance limit.
I expose the patch size and threshold as one "Simplify" slider with a few steps, because the right amount depends on the photo and on how patient the stitcher is.
5. Backgrounds take more thread than the subject
In a typical pet photo, half the chart is carpet, sofa or lawn, and it's the busiest part. So there's an optional focus area: an oval around the subject. Outside it, only the few threads used most there are kept, and a majority vote over a 5 x 5 window smooths them into calm patches. Inside the core of the oval nothing changes, so the face keeps all its detail. Stitches in the subject's colors that reach past the oval (ears, paws) and connect to it are left alone. If the background is already calm (blurred, plain wall), it's skipped entirely.
What I'd tell myself at the start
- Measure against the person using the output. My metrics looked fine long before the charts were pleasant to stitch. Counting single stitches (no same-colored neighbor up, down, left or right) and color changes per row turned out to be better quality numbers than any color error.
- Keep it deterministic. Previews, saved drafts and the paid PDF all have to match exactly.
- Run it on the client. The whole engine is plain TypeScript with no DOM, running in a Web Worker, so the photo never leaves the device and sliders respond instantly. The same code runs in Node for tests.
If you want to see the result, the tool is Picture Cross Stitch. The editor and preview are free in the browser; the printable PDF is the paid part. Happy to answer questions about any of the steps above in the comments.

