Concise way to call ifilter with multiple predicates in python

Question

I'm streaming a very large collection through a script and am currently using ifilter in a simple call to reject certain values, i.e.:

ifilter(lambda x: x in accept_list, read_records(filename))

That's one predicate, but now it's occurred to me I should add another, and I might want to add others in the future. The straightforward way would've been nesting an ifilter call:

ifilter(lambda x : x not in block_list,
    ifilter(lambda x: x in accept_list, read_records(filename)))

I am thinking of simply putting the predicates as unbound functions in a list and using them for this. These repeated ifilter calls seem hard to attain there though (and might not be the best option). Perhaps I can construct a single function that calls all the predicates but how do I write this as concisely (while still readable) as possible?

Answer 1

You could write the following function:

def conjoin(*fns):
    def conjoined(x):
        for fn in fns:
            if not fn(x): return False
        return True
    return conjoined

You would then call it like so:

ifilter(conjoined(lambda x: x not in block_list, lambda x: x in accept_list),
        read_records(filename))

And you could implement a similar disjoin function for or-ing functions together:

def disjoin(*fns):
    def disjoined(x):
        for fn in fns:
            if fn(x): return True
        return False
    return disjoined

There may be a nicer way to implement them, but one has to be careful. One might try applying every function to x and using all or any, but this is undesirable, since using them will require evaluating every predicate on the argument. The solution presented here is satisfactorily short-circuiting.

Also, just for fun, let's implement an invert function

def invert(fn):
    return lambda x: not fn(x)

Now, we have a functionally complete set of function-manipulating functions, and can construct any logical operation out of these :)

Answer 2

I would stick with the following solution:

def complex_predicate(x):
    return x not in block_list and x in accept_list and is_good(x)

or

def complex_predicate2(x):
    return all([x not in block_list, x in accept_list, is_good(x)])

and then

ifilter(complex_predicate, read_records(filename))

or the same with complex_predicate2.

But, I think it is a matter of taste, though.