pythoniteratorpython-itertools

What's the fastest way of skipping tuples with a certain structure in a itertool product?


I have to process a huge number of tuples made by k integers, each ranging from 1 to Max_k.

Each Max can be different. I need to skip the tuples where an element has reached is max value, in that case keeping only the tuple with "1" in the remaining position. The max is enforced by design, so it cannot be that some item is > of its max For example, if the max of the second element of a triple is 4, i need to keep (1,4,1) but skip (1,4,2) , (1,4,3) ... (2,4,1) etc.

I am pretty sure I am missing a much faster way to do that. My typical scenario is tuples with 16 to 20 elements, with maxes in the 50-70 mark. What would be the recommended approach ?

In Python, as a toy example with hardcoded Maxes (5,4,2), is the following:

from itertools import *

def filter_logic(y):
    if y[0]==5:
        if y[1] > 1 or y[2] >1:
            return True
    if y[1]==4:
        if y[0] > 1 or y[2] >1:
            return True
    if y[2]==2:
        if y[0] > 1 or y[1] >1:
            return True
    return False
   
def tuples_all(max_list):
    my_iterables = []
    for limit in max_list:
        my_iterables.append(range(1, limit+1))
    return product(*my_iterables)

def tuples_filtered(max_list):
    return filterfalse(filter_logic, tuples_all(max_list))

max_list = [5,4,2]

print("Original list")
for res in tuples_all(max_list):
    print(res)

print("After filtering")
for fil in tuples_filtered(max_list):
    print(fil)

Output of the filtered tuples:

After filtering
(1, 1, 1)
(1, 1, 2)
(1, 2, 1)
(1, 3, 1)
(1, 4, 1)
(2, 1, 1)
(2, 2, 1)
(2, 3, 1)
(3, 1, 1)
(3, 2, 1)
(3, 3, 1)
(4, 1, 1)
(4, 2, 1)
(4, 3, 1)
(5, 1, 1)

Solution

  • Since you commented that order between tuples isn't important, we can simply produce the tuples with max value and then the tuples without max value:

    from itertools import *
    
    
    def tuples_direct(max_list):
        n = len(max_list)
    
        # Special case
        if 1 in max_list:
            yield (1,) * n
            return
    
        # Tuples with a max.
        for i, m in enumerate(max_list):
            yield (1,) * i + (m,) + (1,) * (n-i-1)
    
        # Tuples without a max.
        yield from product(*(range(1, m) for m in max_list))
    
    
    max_list = [5,4,2]
    for tup in tuples_direct(max_list):
        print(tup)
    

    Output (Attempt This Online!):

    (5, 1, 1)
    (1, 4, 1)
    (1, 1, 2)
    (1, 1, 1)
    (1, 2, 1)
    (1, 3, 1)
    (2, 1, 1)
    (2, 2, 1)
    (2, 3, 1)
    (3, 1, 1)
    (3, 2, 1)
    (3, 3, 1)
    (4, 1, 1)
    (4, 2, 1)
    (4, 3, 1)