I have the following code:
from concurrent.futures import ThreadPoolExecutor
def spam(url, hello=None, params=None):
print(url, hello, params)
urls = [1, 2, 3, 4, 5]
params = [(6, 7), 7, ('a', 1), 9, 'ab']
with ThreadPoolExecutor(5) as executor:
res = executor.map(spam, urls, params)
This expectedly prints:
1 (6, 7) None
2 7 None
3 ('a', 1) None
4 9 None
5 ab None
Is there a way to tell the map
function to call spam
with a particular keyword argument? In this example, I'd like the values params to be passed to the hello
argument rather then the next in line (which in this case is params
).
The real-world use case I'm trying to solve is passing the params=
value to a request.get
call for a repeating URL.
To call this multiple times using threading, I would first create a list of tuples where each tuple is a version of a,b,c: arguments = [(1,2,3), (4,5,6), (7,8,9), ....] That should give you the desired results. Save this answer.
The concurrent.futures module provides a high-level interface for asynchronously executing callables. The asynchronous execution can be performed with threads, using ThreadPoolExecutor , or separate processes, using ProcessPoolExecutor .
ThreadPoolExecutor Methods : submit(fn, *args, **kwargs): It runs a callable or a method and returns a Future object representing the execution state of the method. map(fn, *iterables, timeout = None, chunksize = 1) : It maps the method and iterables together immediately and will raise an exception concurrent.
You could wrap spam
in a lambda when doing map
res = executor.map(lambda x,y:spam(x,params=y), urls, params)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With