Python Pillow - ValueError: Decompressed Data Too Large

Question

I use the Pillow lib to create thumbnails. I have to create a lot of them, actually more than 10.000

The program works fine, but after processing round about 1.500, I get the following error:

    Traceback (most recent call last):
  File "thumb.py", line 15, in <module>
    im = Image.open('/Users/Marcel/images/07032017/' + infile)
  File "/Users/Marcel/product-/PIL/Image.py", line 2339, in open
    im = _open_core(fp, filename, prefix)
  File "/Users/Marcel/product-/PIL/Image.py", line 2329, in _open_core
    im = factory(fp, filename)
  File "/Users/Marcel/product-/PIL/ImageFile.py", line 97, in __init__
    self._open()
  File "/Users/Marcel/product-/PIL/PngImagePlugin.py", line 538, in _open
    s = self.png.call(cid, pos, length)
  File "/Users/Marcel/product-/PIL/PngImagePlugin.py", line 136, in call
    return getattr(self, "chunk_" + cid.decode('ascii'))(pos, length)
  File "/Users/Marcel/product-/PIL/PngImagePlugin.py", line 319, in chunk_iCCP
    icc_profile = _safe_zlib_decompress(s[i+2:])
  File "/Users/Marcel/product-/PIL/PngImagePlugin.py", line 90, in _safe_zlib_decompress
    raise ValueError("Decompressed Data Too Large")
ValueError: Decompressed Data Too Large

My program is very straight forward:

import os, sys
import PIL
from PIL import Image

size = 235, 210
reviewedProductsList = open('products.txt', 'r')
reviewedProducts = reviewedProductsList.readlines()
t = map(lambda s: s.strip(), reviewedProducts)

print "Thumbs to create: '%s'" % len(reviewedProducts)

for infile in t:
    outfile = infile
    try:
        im = Image.open('/Users/Marcel/images/07032017/' + infile)
        im.thumbnail(size, Image.ANTIALIAS)
        print "thumb created"
        im.save('/Users/Marcel/product-/thumbs/' + outfile, "JPEG")
    except IOError, e:
        print "cannot create thumbnail for '%s'" % infile
        print "error: '%s'" % e

I am performing this operation locally on my MacBook Pro.

Answer 1

This is to protect against a potential DoS attack on servers running Pillow caused by decompression bombs. It occurs when a decompressed image is found to have too large metadata. See http://pillow.readthedocs.io/en/4.0.x/handbook/image-file-formats.html?highlight=decompression#png

Here's the CVE report: https:// www.cvedetails.com/cve/CVE-2014-9601/

From a recent issue:

If you set ImageFile.LOAD_TRUNCATED_IMAGES to true, it will suppress the error (but still not read the large metadata). Alternately, you can change set the values here: https://github.com/python-pillow/Pillow/ blob/master/PIL/PngImagePlugin.py#L74

https://github.com/python-pillow/Pillow/issues/2445

Answer 2

following code should help you in setting what accepted answer says.

from PIL import PngImagePlugin
LARGE_ENOUGH_NUMBER = 100
PngImagePlugin.MAX_TEXT_CHUNK = LARGE_ENOUGH_NUMBER * (1024**2)

It's not documented how to set this value. I hope people find this useful.