Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to program to have all processors on your machine used?

I am running a single-threaded python program that performs massive data processing on my windows box. My machine has 8 processors. When I monitor the CPU usage in performance tab under Windows Task Manager, it shows that I am using only a very small fraction of the processing power available to me. Only one processor is being used to the fullest and all the rest are almost idle. What should I do to ensure that all my processors are used? Is multithreading a solution?

like image 748
Aadith Ramia Avatar asked Dec 01 '10 07:12

Aadith Ramia


People also ask

How do I force a program to use all cores?

Core Settings In Windows 10Type 'msconfig' into the Windows Search Box and hit Enter. Select the Boot tab and then Advanced options. Check the box next to Number of processors and select the number of cores you want to use (probably 1, if you are having compatibility issues) from the menu. Select OK and then Apply.

How do I enable all cores?

From the System Utilities screen, select System Configuration > BIOS/Platform Configuration (RBSU) > System Options > Processor Options > Processor Core Disable and press Enter. Enter the number of cores to enable per processor socket and press Enter. If you enter an incorrect value, all cores are enabled.

Is it safe to enable all cores in msconfig?

But don't worry you haven't damaged your CPU but it's recommended that you let the system manage cores automatically. If you enable all of them when Windows doesn't need to you will only drain more power.


2 Answers

multithreading cannot make use of extra processors or cores.

You should spawn new processes instead of new threads.

This tool is by far the simplest among all that I have come across: parallel python

Overview:

PP is a python module which provides mechanism for parallel execution of python code on SMP (systems with multiple processors or cores) and clusters (computers connected via network).

It is light, easy to install and integrate with other python software.

PP is an open source and cross-platform module written in pure python

like image 104
kakarukeys Avatar answered Nov 02 '22 23:11

kakarukeys


Multithreading is required for a single process, but it is not necessarily a solution; processor affinity can restrict it to a subset of available cores even if you have more than enough threads to use all.

like image 40
Ignacio Vazquez-Abrams Avatar answered Nov 03 '22 01:11

Ignacio Vazquez-Abrams