Dither

I keep coming back to John Provencher's work. He takes a photograph, throws away almost all of it, and what's left still reads. The picture comes back rebuilt out of a few hundred marks.

I wanted to understand the rules well enough to dial them myself, so I put everything on a slider. Five ways to reduce a picture, all sharing one pipeline, in a single file with no dependencies.

Every effect is the same three steps

Each one cuts the image into a grid, averages every cell down to one colour and one brightness, then decides what mark to put there. Character map picks a glyph, mark grid picks a shape, halftone picks a dot size, ordered dither picks a palette entry. Only that third step changes, so they all sit behind one function and swapping between them feels like picking up a different tool.

Why I stopped drawing letters

My first go at the mosaic drew everything with fillText, assuming a ramp of solid and hollow squares would cover most of the vocabulary. It didn't. A glyph sits wherever its font metrics put it in the cell, so filled squares never quite meet. An outline gets whatever stroke weight the typeface decided on. A dot is one fixed size forever. All three of those belong to the text renderer and it won't hand them over.

Drawing the marks as vectors got all three back. I kept the characters around as names for shapes, which also let me add triangles and diamonds without worrying about whether a font ships that glyph.

// The characters are tokens, not text. The glyph only has to be typeable
// and suggestive of what it draws.
const MARKS = {
  '■': { kind: 'fill', size: 1 },
  '□': { kind: 'ring', size: 1 },
  '●': { kind: 'dot',  size: 0.86 },
  '▲': { kind: 'tri',  size: 1, up: true },
}

// A filled mark tiles the whole cell, an outline holds a real stroke
// weight, and a dot's radius is free to track tone. Text metrics can do
// none of those.
const inner = cell - gutter * 2
drawMark(ctx, MARKS[token], mx, my, inner, lineWidth, colour)

Halftone dots grow by area

I sized the halftone dots with radius proportional to darkness, which is the obvious way to do it and wrong. The midtones came out heavy and the whole image plugged up. Ink coverage is area, and area goes with the square of the radius, so a cell that's half dark wants a radius of the square root of a half. One missing square root was the whole problem.

const coverage = 1 - luminance / 255

// Area proportional to coverage, so the tonal ramp stays linear.
// Using coverage directly here is the classic mistake: it makes
// every midtone read a stop too dark.
const r = cell * 0.5 * Math.sqrt(coverage) * dotScale

Measuring blur in pixels bit me

Fusing halftone dots into organic blobs is a blur and a re-threshold. It softens the dot mask so neighbours bleed into each other, then cuts it back to hard edges. My first pass took a blur radius in pixels, which seemed harmless until I coarsened the screen. At a wide pitch a fixed radius starts averaging across whole cells when it should only be bleeding between adjacent dots, the midtones flatten out, and the image collapses into a couple of islands.

Expressing it as a fraction of the screen pitch fixed that, and now the control means the same thing at every dot size. I've started checking any parameter I measure in pixels against both ends of its own slider.

Why dithered video boils

Error diffusion pushes each pixel's rounding error into its neighbours, so every pixel depends on the whole path taken to reach it. Nudging the input by one value sends the error down a different route from there on. On a still that's invisible. On footage a flat wall crawls with noise, because two frames that look identical to me aren't identical to the algorithm.

Ordered dither and halftone have no memory. A pixel's threshold comes from its own coordinates, so the same input in the same place gives the same output every frame and flat areas sit perfectly still. That's most of why I reach for those two in motion work, and why I put the warning inside the error diffusion panel where you'd actually hit it. I'd only use Floyd-Steinberg on video if I wanted that texture.

Two ways to export

Recording the canvas live is one click, and it drops frames the moment the filter can't keep up. Stepping the clip frame by frame and writing out a numbered sequence is exact and slow. They're different jobs so I built both. The sequence goes straight into ffmpeg and comes out as ProRes.

One thing I didn't expect. Drawing tens of thousands of glyphs straight onto the on-screen canvas took fifteen seconds. Doing exactly the same work on a detached canvas and blitting the finished frame across once took one and a half. I never touched the algorithm.

Oh yeah and I made it look like early Photoshop because why not?