Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Process a function on different arguments in parallel in Python

This is my simple code where I want to run printRange() in parallel:

def printRange(lrange):
    print ("First is " + str(lrange[0]) + " and last is " + str(lrange[1]))


def runInParallel():
    ranges = [[0, 10], [10, 20], [20, 30]]
    // Call printRange in parallel with each sublist of ranges given as argument

My question is different from this SO question as here, the each process is hardcoded, started and finally joined. I want to run printRange() in parallel with say 100 other printRange() worker functions. Hardcoding each time is not feasible. How could this be done?

like image 828
FlyingAura Avatar asked Jan 02 '23 06:01

FlyingAura


2 Answers

Using multiprocessing

from multiprocessing import Pool


def print_range(lrange):
    print('First is {} and last is {}'.format(lrange[0], lrange[1]))


def run_in_parallel():
    ranges = [[0, 10], [10, 20], [20, 30]]
    pool = Pool(processes=len(ranges))
    pool.map(print_range, ranges)


if __name__ == '__main__':
    run_in_parallel()

Output:

First is 0 and last is 10
First is 10 and last is 20
First is 20 and last is 30
like image 199
balderman Avatar answered Jan 03 '23 19:01

balderman


Something like this?

import threading

def printRange(lrange):
    print ("First is " + str(lrange[0]) + " and last is " + str(lrange[1]))

def runInParallel():
    ranges = [[0, 10], [10, 20], [20, 30]]
    for i in ranges:
        t = threading.Thread(target=printRange, args = [i])
        t.start()

runInParallel()
like image 31
Stef van der Zon Avatar answered Jan 03 '23 20:01

Stef van der Zon