Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the best way in Python to call the same function in separate threads?

What is the best way to call same function in separate threads and have a separate list with returned values for each instance, without duplicating function?

Example:

import threading


def function(a):

    returned_values = []

    ct = threading.currentThread() 
    while getattr(ct, "do_run", True):
        ret = do_something(a)
        returned_values.append(ret)


t1 = threading.Thread(target=function, args=("AAA",))
t2 = threading.Thread(target=function, args=("BBB",))
t3 = threading.Thread(target=function, args=("CCC",))

t1.start()
t2.start()
t3.start()

import time;time.sleep(10)
t1.do_run = t2.do_run = t3.do_run = False

EDIT: Forgot to mention that I use Python 2.7

like image 728
rbs Avatar asked Apr 26 '17 10:04

rbs


People also ask

How do you run the same function on multiple threads in Python?

To implement threading in Python, you have to perform three steps: Inherit the class that contains the function you want to run in a separate thread by using the Thread class. Name the function you want to execute in a thread run() . Call the start() function from the object of the class containing the run() method.

How do you separate threads in Python?

Use the Thread(function, args) to create a new thread. Call the start() method of the Thread class to start the thread. Call the join() method of the Thread class to wait for the thread to complete in the main thread.

Can multiple threads call the same function?

A thread can execute a function in parallel with other threads. Each thread shares the same code, data, and files while they have their own stack and registers.

How do you manage multiple threads in Python?

Use multithreading when you know the program will be waiting around for some external event (i.e., for I/O-bound tasks). Use multiprocessing when your code can safely use multiple cores and manage memory (i.e., for CPU-bound tasks).


1 Answers

Use ThreadPool

Something like this

from multiprocessing.pool import ThreadPool

pool = ThreadPool()
pool.map(function, list_containing_args)

P.S it works similar to multiprocess map.Each argument is given a new thread .You can specify the number of threads you want to spawn if you have limited resources or a big list

from multiprocessing.pool import ThreadPool
import subprocess
def func(ip):
    c=subprocess.Popen("ping -c 3 "+ip, shell=True, stdout=subprocess.PIPE)
    output, error= c.communicate()
    return output

pool = ThreadPool()
for i in  pool.map(func,["127.0.0.1", "www.google.com", "www.facebook.com"]):
    print i
like image 84
vks Avatar answered Oct 03 '22 17:10

vks