Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Globals variables and Python multiprocessing [duplicate]

Possible Duplicate:
Python multiprocessing global variable updates not returned to parent

I am using a computer with many cores and for performance benefits I should really use more than one. However, I'm confused why these bits of code don't do what I expect:

from multiprocessing import Process  var = range(5) def test_func(i):     global var     var[i] += 1  if __name__ == '__main__':     jobs = []     for i in xrange(5):         p = Process(target=test_func,args=(i,))         jobs.append(p)         p.start()  print var 

As well as

from multiprocessing import Pool  var = range(5) def test_func(i):     global var     var[i] += 1  if __name__ == '__main__':     p = Pool()     for i in xrange(5):         p.apply_async(test_func,[i])  print var 

I expect the result to be [1, 2, 3, 4, 5] but the result is [0, 1, 2, 3, 4].

There must be some subtlety I'm missing in using global variables with processes. Is this even the way to go or should I avoid trying to change a variable in this manner?

like image 437
user1475412 Avatar asked Jun 26 '12 20:06

user1475412


People also ask

Can Python multiprocessing access global variables?

You can inherit global variables between forked processes, but not spawned processes. In this tutorial you will discover how to inherit global variables between processes in Python. Learn multiprocessing systematically with this 7-day jump-start.

When two process executing and trying to access the global variable This is a?

A race condition occurs when two threads access a shared variable at the same time.

Can two processes share global variables?

No, since global variables are not shared between processes unless some IPC mechanism is implemented. The memory space will be copied. As a consequence, the global variable in both processes will have the same value inmediately after fork, but if one changes it, the other wont see it changed.

Are global variables Pythonic?

In Python, variables that are only referenced inside a function are implicitly global. If a variable is assigned a value anywhere within the function's body, it's assumed to be a local unless explicitly declared as global.


1 Answers

If you are running two separate processes, then they won't be sharing the same globals. If you want to pass the data between the processes, look at using send and recv. Take a look at http://docs.python.org/library/multiprocessing.html#sharing-state-between-processes for an example similar to what you're doing.

like image 170
John Percival Hackworth Avatar answered Sep 20 '22 11:09

John Percival Hackworth