Using multiple cores with Python and Eventlet

Question

I have a Python web application in which the client (Ember.js) communicates with the server via WebSocket (I am using Flask-SocketIO). Apart from the WebSocket server the backend does two more things that are worth to be mentioned:

• Doing some image conversion (using graphicsmagick)
• OCR incoming images from the client (using tesseract)

When the client submits an image its entity is created in the database and the id is put in an image conversion queue. The worker grabs it and does image conversion. After that the worker puts it in the OCR queue where it will be handled by the OCR queue worker.

So far so good. The WS requests are handled synchronously in separate threads (Flask-SocketIO uses Eventlet for that) and the heavy computational action happens asynchronously (in separate threads as well).

Now the problem: the whole application runs on a Raspberry Pi 3. If I do not make use of the 4 cores it has I only have one ARMv8 core clocked at 1.2 GHz. This is very little power for OCR. So I decided to find out how to use multiple cores with Python. Although I read about the problems with the GIL) I found out about multiprocessing where it says The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads.. Exactly what I wanted. So I instantly replaced the

from threading import Thread
thread = Thread(target=heavy_computational_worker_thread)
thread.start()

by

from multiprocessing import Process
process = Process(target=heavy_computational_worker_thread)
process.start()

The queue needed to be handled by the multiple cores as well So i had to change

from queue import Queue
queue = multiprocessing.Queue()

to

import multiprocessing
queue = multiprocessing.Queue()

as well. Problematic: the queue and the Thread libraries are monkey patched by Eventlet. If I stop using the monkey patched version of Thread and Queue and use the one from multiprocsssing instead then the request thread started by Eventlet blocks forever when accessing the queue.

Now my question:

Is there any way I can make this application do the OCR and image conversion on a separate core?

I would like to keep using WebSocket and Eventlet if that's possible. The advantage I have is that the only communication interface between the processes would be the queue.

Ideas that I already had: - Not using a Python implementation of a queue but rather using I/O. For example a dedicated Redis which the different subprocesses would access - Going a step further: starting every queue worker as a separate Python process (e.g. python3 wsserver | python3 ocrqueue | python3 imgconvqueue). Then I would have to make sure myself that the access on the queue and on the database would be non-blocking

The best thing would be to keep the single process and make it work with multiprocessing, though.

Thank you very much in advance

Answer 1

Eventlet is currently incompatible with the multiprocessing package. There is an open issue for this work: https://github.com/eventlet/eventlet/issues/210.

The alternative that I think will work well in your case is to use Celery to manage your queue. Celery will start a pool of worker processes that wait for tasks provided by the main process via a message queue (RabbitMQ and Redis are both supported).

The Celery workers do not need to use eventlet, only the main server does, so this frees them to do whatever they need to do without the limitations imposed by eventlet.

If you are interested in exploring this approach, I have a complete example that uses it: https://github.com/miguelgrinberg/flack.