Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python multiprocessing.cpu_count() returns '1' on 4-core Nvidia Jetson TK1

Tags:

Can anyone tell me why Python's multiprocessing.cpu_count() function would return 1 when called on a Jetson TK1 with four ARMv7 processors?

>>> import multiprocessing
>>> multiprocessing.cpu_count()
1

The Jetson TK1 board is more or less straight out of the box, and no one has messed with cpusets. From within the same Python shell I can print the contents of /proc/self/status and it tells me that the process should have access to all four cores:

>>> print open('/proc/self/status').read()
----- (snip) -----
Cpus_allowed:   f
Cpus_allowed_list:      0-3
----- (snip) -----

What else could be causing this behavior from cpu_count()?

Edit:

To test Klaus's hypothesis, I used the following code to run a very simple experiment:

import multiprocessing

def f(x):
    n = 0
    for i in xrange(10000):
        n = max(n, multiprocessing.cpu_count())
    return n

p = multiprocessing.Pool(5)
for i in range(10):
    print p.map(f, [1,2,3,4,5])

Which produced the following output:

[3, 3, 3, 3, 1]
[4, 3, 3, 3, 3]
[4, 3, 3, 3, 3]
[3, 3, 4, 3, 3]
[4, 3, 3, 3, 3]
[3, 3, 4, 3, 3]
[4, 3, 3, 3, 3]
[3, 3, 4, 3, 3]
[3, 3, 3, 4, 3]
[4, 3, 3, 3, 3]

Running just a single iteration of p.map(f, [1,2,3,4,5]) usually produces [1, 1, 1, 1, 1], although occasionally a 2 will appear as one of the list elements.

like image 220
Hephaestus Avatar asked Jul 10 '15 15:07

Hephaestus


People also ask

What does multiprocessing cpu_ count return?

cpu_count() We can get the count of the number of CPUs in your system using the multiprocessing. cpu_count() function. This function will return the number of logical CPUs in your system as an integer.

Does Python multiprocessing use multiple cores?

Key Takeaways. Python is NOT a single-threaded language. Python processes typically use a single thread because of the GIL. Despite the GIL, libraries that perform computationally heavy tasks like numpy, scipy and pytorch utilise C-based implementations under the hood, allowing the use of multiple cores.

How do I count CPU in Python?

cpu_count() method in Python is used to get the number of CPUs in the system. This method returns None if number of CPUs in the system is undetermined. Parameter: No parameter is required. Return Type: This method returns an integer value which denotes the number of CPUs in the system.


1 Answers

On Linux systems multiprocessing.cpu_count() relies on a sysconf (_SC_NPROCESSORS_ONLN) call, which returns the number of online CPUs in contrast to sysconf (_SC_NPROCESSORS_CONF) which returns the number of configured CPUs.

The values might differ in systems with advanced CPU power management functionality that sets CPU cores offline to save energy or with similar dynamic CPU activation functionality.

like image 58
Klaus D. Avatar answered Oct 04 '22 15:10

Klaus D.