Why does Pillow convert return colours outside the specified palette?

Question

Using Pillow 5.4.1, Python 3.6.8

Given an image image.png with 9 distinct colours, and given a data palette with 5 distinct colours, one would expect that asking pillow to reduce the image to the described palette that the resulting image would contain colours from only that palette.

However, using the im.im.convert method returns an image with colours outside the specified palette; specifically they are always greyscale images (R==B==G values)

Sample Code, outputting the unique set of colours for the original image, palette, and converted image.

from PIL import Image
im = Image.open("image.png")

# create palette from raw data
# colours: Red, Green, Blue, Black, and White (5 total)
RGBBW = [(255,0,0), (0,255,0), (0,0,255), (0,0,0), (255,255,255)]
data = sum([list(x) for x in RGBBW], [])[:256]
pimg = Image.new("P",(16,16))
pimg.putpalette(data)

# Hack
im.convert("RGB")
cim_ = im.im.convert("P", 0, pimg.im)
cim = im._new(cim_).convert("RGB")

def colors(im):
    cs = []
    for x in range(im.width):
        for y in range(im.height):
            cs.append(im.getpixel((x,y)))
    return list(set(cs))

print("Original: %s" % colors(im))
print("Palette: %s" % RGBBW)
print("Convert: %s" % colors(cim))

Input image: -> <- (3x3 pixel image, all pixels unique colours)

(Larger version, for visualisation only: )

Output:

Original: [(85, 85, 85, 255), (0, 0, 255, 255), (0, 0, 0, 255), (255, 0, 0, 255), (0, 255, 255, 255), (255, 255, 255, 255), (255, 255, 0, 255), (255, 0, 255, 255), (0, 255, 0, 255)]
Palette: [(255, 0, 0), (0, 255, 0), (0, 0, 255), (0, 0, 0), (255, 255, 255)]
Convert: [(252, 252, 252), (0, 0, 255), (255, 0, 0), (0, 0, 0), (170, 170, 170), (0, 255, 0), (84, 84, 84)]

(Note that the hack to prevent dither is a workaround, pending a fix I've contributed to master (yet to be cut into a new release))

The values [(170, 170, 170), (84, 84, 84), (252, 252, 252)] appear in the converted image, but were not specified in the original palette. They all happen to be greyscale.

I think there's something in src/libImaging/Palette.c that's effecting this, but I'm not sure if this is a bug of the code, or a 'feature' of libjpeg

Answer 1

Turns out this issue is both user error and an unexpected initialisation issue.

The initialisation issue: As pointed out in the comments, the palette for a new image is specifically initialised as greyscale.

If we replace the entire palette with our own, then we're fine. Except, I wasn't.

data = sum([list(x) for x in RGBBW], [])[:256]

This line is logically incorrect.

The palette expects a flattened list of up to 256 triples of RGB, that is, an array of max len 768. If the array is anything less than this, then the rest of the greyscale will still be in play.

The better way to re-initialise the palette is to ensure we repeat a value as to override the greyscale.

In this case:

data = (sum([list(x) for x in RGBBW], []) + (RGBBW[-1] * (256 - len(RGBBW))))[:256*3]

That is:

data = (
    sum([list(x) for x in RGBBW], []) # flatten the nested array
    + (RGBBW[-1] * (256 - len(RGBBW))) # extend with the last value, to our required length, if needed
    )[:256*3] # and trim back, if needed.

This will result in the palette always being 768 length.

Using the last value from our provided array is an arbitrary choice, as is only used as a valid padding value.