ImageMagick batch resizing performance

Question

convert \
   original.jpg \
  -quality 85 \
  -colorspace rgb \
  -profile /var/tmp/sRGB.icm \
  -strip \
  -profile /var/tmp/sRGB.icm \
  -filter Lanczos \
  -write mpr:17JPCONV1-original \
  +delete \
mpr:17JPCONV1-original -crop '3000x2001+0+491' -resize '190x126!>' -write thumbWide.jpg +delete \
mpr:17JPCONV1-original -crop '2981x2983+8+0' -resize '75x75!>' -write thumbStandard.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '163x163!>' -write hpSmall.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '1024x1019!>' -write jumbo.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '190x189!>' -write articleInline.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '2048x2037!>' -write superJumbo.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '592x589!>' -write tmagArticle.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '3000x2983!>' -write popup.jpg +delete \
mpr:17JPCONV1-original -crop '2981x2983+8+0' -resize '640x640!>' -write square640.jpg +delete \
mpr:17JPCONV1-original -crop '3000x1689+0+647' -resize '3000x1688!>' -write videoSmall.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '503x500!>' -write slide.jpg +delete \
mpr:17JPCONV1-original -crop '2981x2983+8+0' -resize '151x151!>' -write moth.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2001+0+491' -resize '337x225!>' -write hpMedium.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2001+0+491' -resize '395x264!>' -write sfSpan.jpg +delete \
mpr:17JPCONV1-original -crop '3000x1689+0+647' -resize '3000x1688!>' -write videoLarge.jpg +delete \
mpr:17JPCONV1-original -crop '3000x1689+0+647' -resize '511x288!>' -write hpLarge.jpg +delete \
mpr:17JPCONV1-original -crop '2981x2983+8+0' -resize '320x320!>' -write square320.jpg +delete \
mpr:17JPCONV1-original -crop '3000x1689+0+647' -resize '600x338!>' -write articleLarge.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2001+0+491' -resize '3000x2000!>' -write videoThumb.jpg +delete \
mpr:17JPCONV1-original -crop '2981x2983+8+0' -resize '150x150!>' -write thumbLarge.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '533x530!>' -write blog533.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '151x151!>' -write blogSmallInline.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '362x360!>' -write tmagSF.jpg +delete \
mpr:17JPCONV1-original -crop '2981x2983+8+0' -resize '190x190!>' -write filmstrip.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '480x478!>' -write blog480.jpg +delete \
mpr:17JPCONV1-original -crop '3000x2983+0+0' -resize '427x425!>' -write blog427.jpg +delete \
mpr:17JPCONV1-original -crop '2981x2983+8+0' -resize '50x50!>' -write blogSmallThumb.jpg +delete \
mpr:17JPCONV1-original -crop '3000x1401+0+791' -resize '151x70!>' miniMoth.jpg;

I am trying to generate ~30 crops from an original using one command (using one command is significantly faster than using a single command for each crop). However, this is taking quite a while (~30s) to finish. Any suggestions to speed this up? Can the resize command take advantage of GPUs via OpenCL?

Update:

• Using -thumbnail instead of -resize improves things
• (Thanks to @A R for the tip) Compiling ImageMagick with libjpeg-turbo also improves times by 20%

Answer 1

You should check if your ImageMagick installation comes with OpenCL support:

convert -list configure | grep FEATURES

If it does (like mine), you should see something like this:

FEATURES      HDRI OpenCL

This command

convert -version 

should also give info about supported features.

If it doesn't you should look after getting the most recent version of ImageMagick that has OpenCL support compiled in. Or if you build the package yourself from the sources, make sure OpenCL is used.

---

Update:

Oh wait. There's another feature that could help you, called OpenMP (for multi-processing).

When OpenMP is enabled, ImageMagick commands can execute in parallel on all the cores of your system. So if you have a quad-core system, and resize an image, the resizing happens on 4 cores (or even 8 if you have hyperthreading).

You can now also use the builtin -bench option to make ImageMagick run a benchmark for your command. For example:

convert logo: -resize 500% -bench 10 logo.png
  Performance[1]: 10i 0.689ips 1.000e 14.420u 0:14.510

This command with -resize 500% tells ImageMagick to run the convert command to scale the built-in IM logo: image by 500% in each direction. The -bench 10 part tells it to run that same command 10 times in a loop and then print the performance results:

• Since I don't have OpenMP enabled, I had only 1 thread (Performance[1]:).
• It reports that it ran 10 iterations (10i).
• The speed was nearly 0.7 iterations per second (0.689ips).
• Total user-alotted time was 14.420 seconds.

You should find out how your system is set up regarding resource limits with this command:

identify -list resource
  File       Area     Memory     Map       Disk    Thread         Time
  --------------------------------------------------------------------
   192    4.295GB       2GiB    4GiB  unlimited         1    unlimited

You can see my current system's settings (defaults -- I didn't tweak them). Each of the keywords in the column headers you can use pimp your system.

• The files defines the max concurrently opened files which ImageMagick will use.
• The memory, map, area and disk resource limits are defined in Bytes. For setting them to different values you can use SI prefixes, .e.g 500MB).

If I had OpenMP for ImageMagick on this system, I could run

convert -limit thread 2

in order to enable 2 parallel threads, re-run the benchmark and see if it really makes a difference, and if so how much. The I could set the limit to 4 or even 8 and repeat the excercise....

---

Finally, you could experiment with the internal format of ImageMagick's pixel cache, called MPC (Magick Pixel Cache). Some people say that for large operations the performance improves here, but I have no personal experience with it.

Convert your base picture to MPC first:

convert input.jpeg input.mpc

and only then run:

convert input.mpc [...your long-long-long list of crops...]

and see if this saves you significantly on time.

Most likely you can use this MPC format even "inline" (using the special mpr: notation), similar to how you applied the trick of using the mpr: format (memory program register) that reads the image into a named memory register. But I've never tried this technique to a real world problem, so I can't say how it works out in real life.

---

Update 2:

One more idea:

First check for your exact ImageMagick version: run convert -version.

In case your ImageMagick has a Q16 (or even Q32 or Q64) in its version string (meaning, its internal processes consider all images to have 16bit channel depth, which requires double memory as compared to Q8) -- this is the default nowadays -- you could test what performance benefits you'll achieve by switching to a Q8-build. (You'll pay your performance wins with quality losses, and you'll have to check if you can live with it or not....)

Answer 2

You CPU time is going to 3 tasks:

• JPEG decompression;
• resize;
• JPEG recompression

(Cropping itself takes maybe 1% of your time.)

To decode JPEG, just do it once, hold the result in RAM, and reuse for each output. (Details below.) That way, the cost will be insignificant.

To encode JPEG, use libjpeg-turbo; same API, but a 2-4x speedup if you use x86-{32,64} or ARM hardware.

To resize, ImageMagick is well-known for using ~100x as much CPU as any other software except PhotoShop and GIMP. That includes all photo viewers. It's doing multiple trigonometric functions per pixel, whereas everyone else just does one multiplication. Sometimes, if you look at pixels near a edge in the image, you can see ImageMagick choose a better color than its competitors. But if you think HTML5, Flash, Silverlight, Java, GD (popular with Perl, PHP, and Python web apps) etc look fine, then you don't need such pseudo-AI (artificial intelligence). You might be able to throw GPU (OpenCL) horsepower or more CPU (OpenMP) into ImageMagick, but why bother?

For high-efficiency, the equivalent of ImageMagick (de facto standard) is Imlib2. It is usable from almost as many OS/language environments as ImageMagick.

Your "convert" shell command is equivalent to 10-20 lines of a high level language calling Imlib2: decompress JPEG, and then repeatedly crop, resize, and compress JPEG.

An example without crop (or multiple output) is: Stretch, resize, or thumbnail an image using Perl

Let me know if you want other examples.