Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How does Garbage Collection work with multiple running processes/threads?

I have a server-like Python program that receives packets, and processes them according to the type of packet. To do so, I've spawned multiple processes using the multiprocessing module. I've noticed that garbage collection causes some delay during operation and packets aren't processed within the desired time frame. I know how to disable garbage collection:

import gc
gc.disable()

However my question is how exactly does Python handle garbage collection when there are multiple processes or Threads involved? Are there differences between garbage collection of processes or threads? Do I need to alter the garbage collection for each processes/thread? Or does single change in garbage collection in the parent process/thread take care of handling for all child processes/threads as well?

My current situation uses Python 2.7 however I would be interested to know if it's the same for both Python 2 and Python 3.

like image 831
James Mertz Avatar asked Apr 24 '14 15:04

James Mertz


1 Answers

A process can have several threads. Garbage collection works within one process.

On systems that support fork: If you disable the garbage collection in one process and then fork it (= create a copy of the process) then GC should also be disabled in the copy.

If new processes are created which are not a copy then they have their own garbage collection configuration. By defaut their GC should be turned on.

But there are a lot of libraries which have a Process class. I can not tell what they do. If you use os to spawn a new process (not a copy) then it should run with GC turned on.

The configuration of one process's GC has no effect on the configuration of an other's GC. Processes are boundaries to protect code from oneanother. So anything in one process is not able to reach into an other process, easily.

like image 143
User Avatar answered Oct 06 '22 00:10

User