Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use a thread pool to do infinite loop function?

I want to do a infinite loop function.

Here is my code

def do_request():
    # my code here
    print(result)

while True:
    do_request()

When use while True to do this, it's a little slow, so I want to use a thread pool to concurrently execute the function do_request(). How to do this ?

Just like use ab (Apache Bench) to test HTTP server.

like image 407
Feng Yu Avatar asked Dec 15 '14 06:12

Feng Yu


People also ask

How do you make a thread pool?

To use thread pools, we first create a object of ExecutorService and pass a set of tasks to it. ThreadPoolExecutor class allows to set the core and maximum pool size. The runnables that are run by a particular thread are executed sequentially.

What happens when thread pool is full?

By default, the MaxThreads of the ThreadPool is very high. Usually you'll never get there, your app will crash first. So when all threads are busy the new tasks are queued and slowly, at most 1 per 500 ms, the TP will allocate new threads.

How does thread pool executor works?

ThreadPoolExecutor is an ExecutorService to execute each submitted task using one of possibly several pooled threads, normally configured using Executors factory methods. It also provides various utility methods to check current threads statistics and control them.


1 Answers

Finally, I've solved this problem. I use a variable to limit the thread number.

Here is my final code, solved my problem.

import threading
import time

thread_num = 0
lock = threading.Lock()

def do_request():
    global thread_num
    # -------------
    # my code here
    # -------------
    with lock:
        thread_num -= 1

while True:
    if thread_num <= 50:
        with lock:
            thread_num += 1
        t = threading.Thread(target=do_request)
        t.start()
    else:
        time.sleep(0.01)

Thanks for all replies.

like image 122
Feng Yu Avatar answered Sep 28 '22 11:09

Feng Yu