For my final-year project I added a small "compress before upload" step, and I gave it exactly one setting: re-encode everything as JPEG at quality 80. Photos came out fine. Screenshots did not. Text picked up a grey haze around every character, and the files were bigger than I expected for such simple images. So I flipped it and tried saving photos as 256-colour PNG instead, which made them several times larger. One setting, two kinds of image, two different failures. I wanted to understand why before picking a new rule, so I measured four images.
The setup was deliberately small. Two screenshots I made myself by rendering made-up pages in a browser and capturing them at 1280×800: a Chinese meeting-notes page with only 174 distinct colours, and an admin order table with 1,402. Two photos from Wikimedia Commons, both scaled to 1280 wide: a lake with mountains by Jakub Fryš and a fruit stall by Josep M. Gracia, both CC BY-SA 4.0. Everything was encoded with sharp 0.34.5. The palette PNGs used libimagequant through sharp with a 256-colour cap.
Screenshots break the idea that lossy means smaller
The meeting-notes screenshot was 313,983 bytes as a lossless PNG. As JPEG quality 60 it was 123,658 bytes, and at quality 80 it was 164,265 bytes. As a palette PNG it was 116,231 bytes, smaller than both JPEGs and identical to the original pixel for pixel. That last part surprised me until I thought about what a palette PNG is. It stores a table of up to 256 colours, then one index per pixel, then compresses those indices losslessly. If the whole image only has 174 colours, nothing has to be approximated at all. Large white areas and one or two text colours also mean long runs of the same index, which is exactly what lossless compression is good at. In this screenshot 87.9% of pixels had the same colour as their right-hand neighbour.
The table screenshot told the same story: 58,317 bytes as a palette PNG versus 111,650 bytes as JPEG quality 80.
The dirty edges come from how JPEG works, at least as far as I understand it from reading about it. It splits the image into 8×8 blocks and stores each block as frequency components, keeping the low frequencies carefully and the high ones roughly. A sharp black-on-white glyph edge is almost all high frequency, so trimming it leaves ripples around the text. JPEG also stores colour at half resolution, which is why the coloured status badges in the table screenshot got halos. WebP at quality 80 had far less noise and came in at 112,844 bytes, about the same as the palette PNG, but it is still lossy and the glyph edges were slightly soft.
Photos: the palette runs out
For photos everything reversed. The lake photo has 119,846 distinct colours. As a 256-colour palette PNG it was 447,369 bytes, while JPEG quality 80 was 117,320 bytes, so 3.8 times larger. The fruit stall was 2.5 times larger. Only 7.4% of the lake's pixels matched their right neighbour, so the lossless stage had very little repetition to work with.
Size was not the only problem. In a 420×260 crop of the sky, the original had 20,345 colours and the palette version had 148. With dithering off, the smooth cloud gradient turned into visible bands of flat colour. With dithering on, the bands broke up into fine grain that looks fuzzy up close. JPEG at quality 80 showed neither.
I had assumed every photo would band like that. It doesn't. A misty area lower in the same photo looked fine at 256 colours, because it is basically one blue-grey hue and 256 shades of it are enough. So the honest version is: photos with lots of colour and large smooth gradients are the ones that band.
So a compressor has to guess what the image is
I wanted to see how an existing tool handles this, so I used ImgIng (https://imging.ai/) in Chromium 149 and only looked at the defaults it picked. The text screenshot was detected as an icon/line-art type image and defaulted to 8-bit PNG, producing a 174-colour file identical to the original. The lake photo was detected as a photo and defaulted to WebP at quality 75. The fruit stall, which has a lot of fine detail, was classified as a screenshot/UI image and got quality 84 instead, though it stayed WebP. I haven't read its code, so I can only guess it looks at colour count or how concentrated the colours are.
My project now counts colours first: 256 or fewer goes to palette PNG, everything else to WebP. That threshold is crude. The table screenshot has 1,402 colours and still reaches 62 dB PSNR as a 255-colour palette PNG, yet my rule sends it to WebP. I haven't settled on a better cutoff. If you want to know which side your own images fall on, export one as a palette PNG and as JPEG 80, compare the sizes, then zoom in on any text.
