Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I send data to a running python thread?

I have a class that is run in separate threads in my application. I can have multiple threads running at a time and the threads are daemons. After a period of time, some of these threads need to receive and process a message. How do I do this?

A sample of my code looks like this:

import threading
import time 

class MyThread(threading.Thread):
    def __init__(self, args=(), kwargs=None):
        threading.Thread.__init__(self, args=(), kwargs=None)
        self.daemon = True
        self.receive_messages = args[0]

    def run(self):
        print threading.currentThread().getName(), self.receive_messages

    def do_thing_with_message(self, message):
        if self.receive_messages:
            print threading.currentThread().getName(), "Received %s".format(message)

if __name__ == '__main__':
    threads = []
    for t in range(10):
        threads.append( MyThread(args=(t % 2 == 0,)))
        threads[t].start()
        time.sleep(0.1)

    for t in threads:
        t.do_thing_with_message("Print this!")

This outputs:

Thread-1 True
Thread-2 False
Thread-3 True
Thread-4 False
Thread-5 True
Thread-6 False
Thread-7 True
Thread-8 False
Thread-9 True
Thread-10 False
MainThread Received %s
MainThread Received %s
MainThread Received %s
MainThread Received %s
MainThread Received %s

I am expecting, however, those last five lines to not be related to the MainThread, and instead of %s, I'd expect it to me Print this!, like so:

Thread-1 True
Thread-2 False
Thread-3 True
Thread-4 False
Thread-5 True
Thread-6 False
Thread-7 True
Thread-8 False
Thread-9 True
Thread-10 False
Thread-1 Received Print this!
Thread-3 Received Print this!
Thread-5 Received Print this!
Thread-7 Received Print this!
Thread-9 Received Print this!

How can I properly send a message like this to the running threads?

Addendum:

If I have this block after the Print this! block, and utilize @dano's code to solve the problem above, it does not seem to respond to these new messages.

for t in threads:
    t.queue.put("Print this again!")
    time.sleep(0.1)

In this case, I'd expect the end of my output to look like this

Thread-1 Received Print this!
Thread-3 Received Print this!
Thread-5 Received Print this!
Thread-7 Received Print this!
Thread-9 Received Print this!
Thread-1 Received Print this again!
Thread-3 Received Print this again!
Thread-5 Received Print this again!
Thread-7 Received Print this again!
Thread-9 Received Print this again!
like image 258
Python Novice Avatar asked Sep 18 '14 05:09

Python Novice


People also ask

How do you share data across a thread in Python?

You can protect data variables shared between threads using a threading. Lock mutex lock, and you can share data between threads explicitly using queue. Queue.

How can we share data between the threads?

All static and controlled data is shared between threads. All other data can also be shared through arguments/parameters and through based references, as long as the data is allocated and is not freed until all of the threads have finished using the data.

How do you retrieve data from a thread in Python?

For example: def foo(bar, result, index): print 'hello {0}'. format(bar) result[index] = "foo" from threading import Thread threads = [None] * 10 results = [None] * 10 for i in range(len(threads)): threads[i] = Thread(target=foo, args=('world! ', results, i)) threads[i].

How do you call a thread in Python?

Use the Thread(function, args) to create a new thread. Call the start() method of the Thread class to start the thread. Call the join() method of the Thread class to wait for the thread to complete in the main thread.


1 Answers

You can use a Queue.Queue (or queue.Queue in Python 3) for this:

import threading
import time 
from Queue import Queue

print_lock = threading.Lock()

class MyThread(threading.Thread):
    def __init__(self, queue, args=(), kwargs=None):
        threading.Thread.__init__(self, args=(), kwargs=None)
        self.queue = queue
        self.daemon = True
        self.receive_messages = args[0]

    def run(self):
        print threading.currentThread().getName(), self.receive_messages
        val = self.queue.get()
        self.do_thing_with_message(val)

    def do_thing_with_message(self, message):
        if self.receive_messages:
            with print_lock:
                print threading.currentThread().getName(), "Received {}".format(message)

if __name__ == '__main__':
    threads = []
    for t in range(10):
        q = Queue()
        threads.append(MyThread(q, args=(t % 2 == 0,)))
        threads[t].start()
        time.sleep(0.1)

    for t in threads:
        t.queue.put("Print this!")

    for t in threads:
        t.join()

We pass a Queue instance to each thread, and send our message to the Thread using queue.put. We wait for the message to arrive in the run method, which is the part of the Thread object that's actually running in a separate thread of execution. Once we get the message, we call do_thing_with_message, which will run in the same background thread.

I've also added a threading.Lock to the code so the prints to stdout don't get mixed up.

Edit:

If you want to be able to deliver multiple messages to the thread, just use a loop:

def run(self):
    print threading.currentThread().getName(), self.receive_messages
    while True:
        val = self.queue.get()
        if val is None:   # If you send `None`, the thread will exit.
            return
        self.do_thing_with_message(val)
like image 55
dano Avatar answered Oct 07 '22 12:10

dano