Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to batch convert 1000's of images across multiple sub directories in parallel using Image Magic

I have ~100 subdirectories each having ~1000 files I want to convert JPG to PNG using Image Magick under BASH for Win10 i.e. LINUX script. My script is slow, can I speed it up?

find . -type f -name '*.jpg' -exec sh -c '
    orgfile="$0"
    newfile="$(echo "$0" | sed 's/.jpg/.png/')"
    echo $orgfile $newfile
    convert $orgfile -unsharp 0x5 $newfile
    rm $orgfile
' {} \;

I like the loop process because the convert is the first in a number of processes so the input and output names can be reused. However its slow and echo is there for feedback (change to per dir?)

In a related post the following solution is given

# Runs these conversions serially
ls *.NEF | sed 's#.NEF##' | xargs -I^ convert ^.NEF ^.jpg
# Runs these conversions with 8 different processes
ls *.NEF | sed 's#.NEF##' | xargs -P8 -I^ convert ^.NEF ^.jpg

But another post warns that parallel processing may slow down a system

/media/ramdisk/img$ time for f in *.bmp; do echo $f ${f%bmp}png; done | xargs -n 2 -P 2 convert -auto-level

I think I'm getting lost in both the advanced BASH scripting and parallel processing and I have no idea of xargs.

BTW running serially is using about 25% of PC resources

like image 347
Miles Avatar asked Oct 17 '22 21:10

Miles


1 Answers

If mogrify only uses 1 CPU you can parallelize using GNU Parallel:

parallel mogrify -unsharp 0x5 -format png ::: *.jpg

Or if the file list is too long for the shell:

ls | parallel mogrify -unsharp 0x5 -format png {} 

Multiple subdirs:

find subdir1 subdir2 -name '*.jpg' | parallel mogrify -unsharp 0x5 -format png {} 
like image 137
Ole Tange Avatar answered Nov 01 '22 08:11

Ole Tange