Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Do Docker containers share a single Python GIL?

When I run a Python script inside a Docker container, it completes one execution loop in ~1 minute. Now as I spin up 2 more containers from same image, and run Python scripts inside them, everything slow down to a crawl and start requiring 5-6 minutes per loop.

None of the scripts are resource bound; there is plenty of RAM and CPU cores sitting around idle. This happens when running 3 containers on a 64-core Xeon Phi system.

So does Docker share a common Python GIL lock among all containers? What are my options to separate the GILs, so each process will run at its full potential speed?

Thank you!

like image 907
Kingua Avatar asked Oct 30 '22 09:10

Kingua


1 Answers

So does Docker share a common Python GIL lock among all containers?

NO.

The GIL is per Python process, a Docker container may have 1 or many Python processes, each with it's own GIL.

If you are not multi-threading, you should not even be aware of the GIL. Are you using threads at all?

like image 118
aaa90210 Avatar answered Nov 15 '22 05:11

aaa90210