Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Iterate over itertools.product in different order without ever creating list

I have an iterable that covers a huge search space. My plan is not to let the script terminate, but to just kill it after a certain amount of time.

Now I need the Cartesian product of this space and search in there. itertools.product produces this order:

>>> list(itertools.product(range(3), repeat=2))
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)]

While I want to search in a diagonal order similar to:

[(0, 0), (0, 1), (1, 0), (0, 2), (1, 1), (2, 0), (1, 2), (2, 1), (2, 2)]

sorted with some key function that returns the sum of the elements of the tuple would be my regular approach, but for sorting all data needs to be inspected, which is infeasible in my case. Is there a way to do this?

This question is very similar to this one, but there sorted is still used in the answer. Also I don't quickly see how to adapt ordered_combinations to ordered_product.

like image 322
qpllb Avatar asked Oct 07 '16 12:10

qpllb


People also ask

Is Itertools faster than for loops?

That being said, the iterators from itertools are often significantly faster than regular iteration from a standard Python for loop.

What is itertool?

Itertools is a module in Python, it is used to iterate over data structures that can be stepped over using a for-loop. Such data structures are also known as iterables. This module works as a fast, memory-efficient tool that is used either by themselves or in combination to form iterator algebra.

Is itertools product efficient?

Conclusion. In conclusion, itertools. product() makes our code efficient, concise, and pythonic.

Do I need to import Itertools?

This module incorporates functions that utilize computational resources efficiently. Using this module also tends to enhance the readability and maintainability of the code. The itertools module needs to be imported prior to using it in the code.


1 Answers

The question is equivalent to asking how to create all n-tuples for with a given sum for successive and increasing values of the sum:

                  (0, 0),               sum == 0
              (0, 1), (1, 0),           sum == 1
        (0, 2), (1, 1), (2, 0),         sum == 2
             (1, 2), (2, 1),            sum == 3
                  (2, 2)                sum == 4

For any given row (with a given target sum), the subproblem is equivalent to the dynamic programming problem Number of ways to make change for amount N or Number of ways to add up to a sum S with N numbers .

Also see the write ups in Combinatorial Algorithms by Donald Knuth.

like image 72
Raymond Hettinger Avatar answered Sep 24 '22 13:09

Raymond Hettinger