Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How many parallel processes?

I am running some code in parallel by using a forking module in perl called Parallel::ForkManager. I have currently setting the maximum number of processes to 30:

my $pm = Parallel::ForkManager->new(30);

What would be an advisable maximum number of processes to create? I am doing this on a commercial grade Solaris server, but I still don't want to overload the system.

like image 674
CJ7 Avatar asked Mar 01 '16 23:03

CJ7


People also ask

How many parallel processes should I run?

You can create concurrent solutions and execute them on a system with a only a single CPU. Parallelism refers to the ability to execute two or more concurrent processes simultaneously. You must have more than one processing core to execute two processes in parallel.

How many types of parallel processing are there?

There are two types of parallel processes: fine-grained and coarse-grained.

How many processes can run simultaneously?

A multitasking operating system may just switch between processes to give the appearance of many processes executing simultaneously (that is, in parallel), though in fact only one process can be executing at any one time on a single CPU (unless the CPU has multiple cores, then multithreading or other similar ...


1 Answers

In downloading files, this really depends on

  1. how many different hosts you're downloading from, and
  2. how fast they will give you the requested files compared to your maximum bandwidth.

If you're downloading files from a single machine to a single machine on a local network, 2-3 is about max. If you're downloading files from 30 different servers on the internet, all of which are slow, but you have a fat pipe, then 30 might be reasonable.

There is no one universal right answer here. Unless you count "it depends."

like image 90
Tanktalus Avatar answered Nov 15 '22 21:11

Tanktalus