Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Non blocking python process or thread

Tags:

python

I have a simple app that listens to a socket connection. Whenever certain chunks of data come in a callback handler is called with that data. In that callback I want to send my data to another process or thread as it could take a long time to deal with. I was originally running the code in the callback function, but it blocks!!

What's the proper way to spin off a new task?

like image 358
coryjacobsen Avatar asked Jul 13 '11 18:07

coryjacobsen


People also ask

Is Python blocking or nonblocking?

But whereas in concurrency, we used a single hand (one thread or process) to make two requests. Usually Python will wait for the response to reach back and then proceeds with sending the next one. This is called Blocking operation. When we do concurrency tasks, we are making the Python code do Non-blocking operation.

How do you block a thread in Python?

One thread can block when waiting for another thread to terminate. This is achieved by the waiting thread calling the join() function on the other running thread. This function call will block until the other thread finishes, or returns immediately if the thread has already terminated.

What is non-blocking thread?

In computer science, an algorithm is called non-blocking if failure or suspension of any thread cannot cause failure or suspension of another thread; for some operations, these algorithms provide a useful alternative to traditional blocking implementations.


2 Answers

threading is the threading library usually used for resource-based multithreading. The multiprocessing library is another library, but designed more for running intensive parallel computing tasks; threading is generally the recommended library in your case.

Example

import threading, time def my_threaded_func(arg, arg2):     print "Running thread! Args:", (arg, arg2)     time.sleep(10)     print "Done!"  thread = threading.Thread(target=my_threaded_func, args=("I'ma", "thread")) thread.start() print "Spun off thread" 
like image 123
stefan Avatar answered Sep 23 '22 22:09

stefan


The multiprocessing module has worker pools. If you don't need a pool of workers, you can use Process to run something in parallel with your main program.

like image 31
nmichaels Avatar answered Sep 23 '22 22:09

nmichaels