How can I save a huge image with PIL?

Question

I want to assemble 8 images of 9450x75600 px (i.e. 75600x75600 px)

import sys
import PIL
from PIL import Image
    
PIL.Image.MAX_IMAGE_PIXELS = 9331200000
ListeImage=['test1.tif','test2.tif','test3.tif','test4.tif','test5.tif','test6.tif','test7.tif','test8.tif']
images = [Image.open(x) for x in ListeImage]
widths, heights = zip(*(i.size for i in images))
    
total_width = sum(widths)
max_height = max(heights)
    
new_im = Image.new('RGB', (total_width, max_height))
    
y_offset = 0
for im in images:
      new_im.paste(im, (0,y_offset))
      y_offset += im.size[0]
      new_im.save('TOTAL'+str(y_offset)+'.tif')

but I've got this error...

Traceback (most recent call last):
  File "C:\Python27\MergeImages.py", line 21, in <module>
    new_im.save('test'+str(bande[0])+'.tif')
  (...)
  File "C:\Python27\lib\site-packages\PIL\TiffImagePlugin.py", line 626, in _pack
    return struct.pack(self._endian + fmt, *values)
error: integer out of range for 'L' format code

I think it is a memory problem. How to solve it?

Answer 1

You are getting an exception because you are exceeding the 4GB Tiff format limit.

See: What is the maximum size of TIFF metadata?.

You may use BigTIFF format.
You may try using Tifffile for writing BigTIFF image files.

I prefer using JPEG 2000 image format.
I found this post regarding saving JPEG 2000 with pillow.

The way I know is saving JPEG 2000 using OpenCV.
OpenCV saves JP2 images in lossless format (same way as Tiff is lossless).

Issues saving with OpenCV:

• We need to convert pillow image to NumPy array.
• We need to convert from RGB to BGR color format.

Here is a modified version of your code that saves JP2 using OpenCV:

import sys
import PIL
from PIL import Image
import cv2
import numpy as np
    
PIL.Image.MAX_IMAGE_PIXELS = 9331200000
ListeImage=['test1.tif','test2.tif','test3.tif','test4.tif','test5.tif','test6.tif','test7.tif','test8.tif']
images = [Image.open(x) for x in ListeImage]
widths, heights = zip(*(i.size for i in images))
    
#total_width = sum(widths)
#max_height = max(heights)
    
#new_im = Image.new('RGB', (total_width, max_height))
new_im = Image.new('RGB', (max(widths), sum(heights)))
    
y_offset = 0
for im in images:
    new_im.paste(im, (0,y_offset))
    y_offset += im.size[1] #im.size[0]
    # new_im.save('TOTAL'+str(y_offset)+'.tif')
    cv2.imwrite('TOTAL'+str(y_offset)+'.jp2', cv2.cvtColor(np.array(new_im), cv2.COLOR_RGB2BGR))

---

Notes:

• It looks like you are mixing width and height - I tried to fix it.
• I don't have enough RAM in my system for testing the code with such large images - I tested it with smaller images.
• I tested the code using Python 3.6, and I don't know if it's going to work with Python 2.7
• I implemented the code without intermediate variables hoping it consumes less RAM.

---

Update:

The above solution consumes way too much RAM (more than 100GB).
Using a large page file (disk space as virtual memory) works, but it's too slow.
The solution (saving as JPEG 2000) is not a practical for most systems.

The solution below uses BigTIFF format.
The implementation is also more efficient in terms of RAM:

import sys
import PIL
from PIL import Image
import tifffile
import numpy as np
import gc
    
PIL.Image.MAX_IMAGE_PIXELS = 9331200000
ListeImage=['test1.tif','test2.tif','test3.tif','test4.tif','test5.tif','test6.tif','test7.tif','test8.tif']
images = [Image.open(x) for x in ListeImage]
widths, heights = zip(*(i.size for i in images))

# Free memory - release memory of all images.
del images
gc.collect()  # Explicitly invoke the Garbage Collector https://stackoverflow.com/questions/1316767/how-can-i-explicitly-free-memory-in-python

    
#new_im = Image.new('RGB', (max(widths), sum(heights)))
new_im = np.zeros((sum(heights), max(widths), 3), np.uint8)  # Use NumPy array instead of pillow image.
    
y_offset = 0
for x in ListeImage:
    im = Image.open(x)  # Read one input image at a time (for saving RAM).
    #new_im.paste(im, (0,y_offset))
    new_im[y_offset:y_offset+im.size[1], :, :] = np.array(im)  # Copy im to NumPy array (instead of pasting to pillow image - saves RAM).
    y_offset += im.size[1]
    tifffile.imwrite('TOTAL'+str(y_offset)+'.tif', new_im, bigtiff=True)  # Write new_im as BigTIFF.

    del im
    gc.collect()  # Explicitly invoke the Garbage Collector