Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Any way to speedup itertool.product

I am using itertools.product to find the possible weights an asset can take given that the sum of all weights adds up to 100.

min_wt = 10
max_wt = 50
step = 10
nb_Assets = 5

weight_mat = []
for i in itertools.product(range(min_wt, (max_wt+1), step), repeat = nb_Assets):
    if sum(i) == 100:
        weight = [i]
        if np.shape(weight_mat)[0] == 0:
            weight_mat = weight
        else:
            weight_mat = np.concatenate((weight_mat, weight), axis = 0)

The above code works, but it is too slow as it goes through the combinations that are not acceptable, example [50,50,50,50,50] eventually testing 3125 combinations instead of 121 possible combinations. Is there any way we can add the 'sum' condition within the loop to speed things up?

like image 434
abhay Samudrasok Avatar asked Nov 12 '19 17:11

abhay Samudrasok


People also ask

Is Itertools product faster?

That being said, the iterators from itertools are often significantly faster than regular iteration from a standard Python for loop.

Is Itertools product faster than nested for loop?

Example of a double loop with 1000 elements: The result of itertools. product() is faster to unpack. Nested loops are about the same (slightly faster) as itertools.

Is itertools important?

Itertools is a module in Python, it is used to iterate over data structures that can be stepped over using a for-loop. Such data structures are also known as iterables. This module works as a fast, memory-efficient tool that is used either by themselves or in combination to form iterator algebra.


1 Answers

Many improvements are possible.

For starters, the search space can be reduced using itertools.combinations_with_replacement() because summation is commutative.

Also, the last addend should be computed rather than tested. For example if t[:4] was (10, 20, 30, 35), you could compute t[4] as 1 - sum(t), giving a value of 5. This will give a 100-fold speed-up over trying one-hundred values of x in (10, 20, 30, 35, x).

like image 50
Raymond Hettinger Avatar answered Sep 28 '22 02:09

Raymond Hettinger