Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python multiprocessor programming

In python, is there a way to find out which CPU a process is running on? For example if I create different processes for different tasks, using the multiprocessing module, is it possible to identify the core in which each process is running?

like image 682
s_hamed Avatar asked Nov 29 '11 07:11

s_hamed


People also ask

Can you do multiprocessing in Python?

The Python multiprocessing module allows you to create and manage new child processes in Python.

How does Python handle multiprocessing?

In this example, at first we import the Process class then initiate Process object with the display() function. Then process is started with start() method and then complete the process with the join() method. We can also pass arguments to the function using args keyword.

Is Python truly multithreaded?

Python doesn't support multi-threading because Python on the Cpython interpreter does not support true multi-core execution via multithreading. However, Python does have a threading library. The GIL does not prevent threading.


2 Answers

Short answer

No, it is not possible.

Long answer

in a general situation you cannot do that since processes are not bound to specific cores. Processes do not have fixed CPUs that they are always guaranteed to run on: it is up to the operating system to decide, which core it uses for running a specific process on a specific time. This decision making is called scheduling and its implementation is OS specific.

On specific operating systems, you may be able to control, how processors are used for excution of specific processes. The assignment of preferred processors is often referred to as processor affinity. Even setting affinitity does not guarantee that a process will always be executed on given cores: it is ultimately up to the OS (and CPU) to decide how the scheduling is ultimately executed.

From all OSes I know, the closest thing I could think of would be Linux's sched_getcpu which can be used "to determine the CPU on which the calling thread is running" (see man sched_getcpu). Even given that you check current CPU with this function, the kernel may change the core right after.

like image 187
jsalonen Avatar answered Nov 10 '22 00:11

jsalonen


I don't think this can be done reliably as a process is not limited to a core. One process can execute on one or more cores (if it uses threads) and the cores that are used to execute it can change over time as the os tries to balance the workload.

As for a nice way to get process-related information look in to the psutil library.

like image 26
Toni Ruža Avatar answered Nov 10 '22 00:11

Toni Ruža