ImageMagick/Paperclip - Paperclip fails during large job

Question

I'm using Paperclip with ImageMagick to process a few thousands images and upload them to S3. It goes through hundreds of images with no issues, and then gives this error:

An error was received while processing: #<Paperclip::Errors::CommandNotFoundError: Could not run theidentifycommand. Please install ImageMagick>

I've seen other questions referencing this error, but it seems strange to me that it can process fine one second, and then the next not be able to find ImageMagick. I'm processing with Sidekiq with 25 concurrent workers.

It's interesting to note that when shutting down sidekiq and restarting, it starts to work again.

Answer 1

I was having the exact same problem, and have traced it down to a thread-safety issue in Paperclip. See https://github.com/thoughtbot/paperclip/issues/1709