Is itertools thread-safe?

Question

For instance, if I create an iterator using chain, can I call it on multiple threads? Note that thread-safety that relies on the GIL is acceptable, but not preferable.

(Note that this is a bit different from this question, which deals with generators, not iterators written in C).

Answer 1

Firstly, nothing in the official documentation on itertools say that they're thread-safe. So it seems that by specification Python does not guarantee anything about that. This might be different across implementations such as Jython or PyPy, but this means your code probably wont be portable.

Secondly, most itertools (with exception of simple ones, like count) take other iterators as their input. You'd need these iterators to also behave correctly in a thread-safe way.

Thirdly, some iterators might not make sense when used simultaneously by different threads. For example izip working in multiple threads might get into race condition taking elements from multiple sources, especially as defined by equivalent python code (what will happen when one thread will manage to take value from only one input iterator, then second thread from two of them?).

Also note that the documentation does not mention that itertools are implemented in C. We know (as an implementation detail) that CPython's itertools are actually written in C, but on other implementations they can happily be implemented as generators, and you can go back to the question you cited.

So, no, you cannot assume that they are thread-safe unless you know implementation details of your target python platform.