Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python multiprocessing keyword arguments

Here is a simple example of using keyword arguments in a function call. Nothing special.

def foo(arg1,arg2, **args):
    print arg1, arg2
    print (args)
    print args['x']

args ={'x':2, 'y':3}
foo(1,2,**args)

Which prints, as expected:

1 2
{'y': 3, 'x': 2}
2

I am trying to pass the same style keyword arguments to a multiprocessing task, but the use of **, in the args list is a syntax error. I know that my function, stretch() will take two positional arguments and n keyword arguments.

pool = [multiprocessing.Process(target=stretch, args= (shared_arr,slice(i, i+step),**args)) for i in range (0, y, step)]

Is it possible to pass keyword arguments to a multiprocessing.Process? If so, how? If not, why?

like image 667
Jzl5325 Avatar asked Jul 18 '12 17:07

Jzl5325


People also ask

How do you pass arguments in multiprocessing in Python?

Passing Keyword Arguments to Multiprocessing Processes We can also pass in arguments corresponding to the parameter name using the kwargs parameter in the Process class. Instead of passing a tuple, we pass a dictionary to kwargs where we specify the argument name and the variable being passed in as that argument.

What is pool in multiprocessing Python?

The Pool class in multiprocessing can handle an enormous number of processes. It allows you to run multiple jobs per process (due to its ability to queue the jobs). The memory is allocated only to the executing processes, unlike the Process class, which allocates memory to all the processes.

How do you use multiprocessing in Python?

In this example, at first we import the Process class then initiate Process object with the display() function. Then process is started with start() method and then complete the process with the join() method. We can also pass arguments to the function using args keyword.

How do you return a value from a Python function?

To get the return value of a function passed to Python multiprocessing. Process, we can use the manager. dict method to create a shared variable. to create a Process object with the target set to the worker function that runs for each process.


1 Answers

The dictionary you are using as keyword args should be passed in as the kwargs parameter to the Process object.

pool = [multiprocessing.Process(target=stretch, args= (shared_arr,slice(i, i+step)),kwargs=args) for i in range (0, y, step)]
like image 151
Andrew Cox Avatar answered Oct 14 '22 15:10

Andrew Cox