Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use a map with *args to unpack a tuple in a python function call

I am currently doing a merge over a set of variables that I'd like to parallelize. My code looks something like this:

mergelist = [
  ('leftfile1', 'rightfile1', 'leftvarname1', 'outputname1'),
  ('leftfile1', 'rightfile1', 'leftvarname2', 'outputname2')
  ('leftfile2', 'rightfile2', 'leftvarname3', 'outputname3')
]

def merger(leftfile,rightfile,leftvarname,outvarname):
   do_the_merge

for m in mergelist:
     merger(*m)

Ordinarily, to speed up long loops, I would replace the for m in mergelist with something like....

from multiprocessing import Pool

p = Pool(8)
p.map(merger(m), mergelist)
p.close()

But since I'm using the star to unpack the tuple, it's not clear to me how to map this correctly. How do I get the *m?

like image 436
Mittenchops Avatar asked Feb 26 '14 20:02

Mittenchops


1 Answers

Use lambda:

with Pool(8) as p:
    p.map(lambda m:merger(*m), mergelist)
like image 161
ndpu Avatar answered Sep 19 '22 08:09

ndpu