I have some code that looks like
with futures.ThreadPoolExecutor(max_workers=2) as executor:
for function in functions:
executor.submit(function)
How would I log which function is currently being handled by the executor? I may or may not have the capability to log from within the functions - would want the executor itself to log something like
print "handling process {i}".format(i=current_process)
Any thoughts on how to approach this?
The Python ThreadPoolExecutor allows you to create and manage thread pools in Python. Although the ThreadPoolExecutor has been available since Python 3.2, it is not widely used, perhaps because of misunderstandings of the capabilities and limitations of Threads in Python.
ThreadPoolExecutor Thread-Safety Although the ThreadPoolExecutor uses threads internally, you do not need to work with threads directly in order to execute tasks and get results. Nevertheless, when accessing resources or critical sections, thread-safety may be a concern.
The ThreadPoolExecutor is designed to speed-up your program by executing tasks concurrently. Nevertheless, in some use cases, using the ThreadPoolExecutor can make your program slower. Sometimes dramatically slower than performing the same task in a for loop.
How to create a ThreadPoolExecutor? With the help of concurrent. futures module and its concrete subclass Executor, we can easily create a pool of threads. For this, we need to construct a ThreadPoolExecutor with the number of threads we want in the pool.
I guess this is a little old but I stumbled across the questions and thought I would put an answer in. I just used a wrapper that can reference an instance of a logger prior to calling the function:
import logging
import os
import concurrent.futures
logging.basicConfig(filename=os.path.expanduser('~/Desktop/log.txt'), level=logging.INFO)
logger = logging.getLogger("MyLogger")
def logging_wrapper(func):
def wrapped(*args, **kwargs):
logger.info("Func name: {0}".format(func.__name__))
func(*args, **kwargs)
return wrapped
def a():
print('a ran')
def b():
print('b ran')
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
for func in [a, b]:
executor.submit(logging_wrapper(func))
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With