Is there a more elegant way to filter the failed results of a function?

Question

For now I have something in my code that looks like this:

def f(x):
    if x == 5:
        raise ValueError
    else:
        return 2 * x

interesting_values = range(10)
result = []
for i in interesting_values:
    try:
        result.append(f(i))
    except ValueError:
        pass

f is actually a more complex function and it fails for specific values in an unpredictible manner (I can't know if f(x) will fail or not before trying it).

What I am interested in is to have this result: the list of all the valid results of f.

I was wondering if there is a way to make the second part like a list comprehension. Of course I can't simply do this:

def f(x):
    if x == 5:
        raise ValueError
    else:
        return 2 * x

interesting_values = range(10)
result = [f(i) for i in interesting_values]

because the call for f(5) will make everything fail, but maybe there is a way to integrate the try-except structure in a list comprehension. Is it the case?

EDIT: I have control over f.

Answer 1

It seems like you have control of f and can modify how it handles errors.

If that's the case, and None isn't a valid output for the function, I would have it return None on an error instead of throwing:

def f(x):
    if x == 5: return None
    else: return 2*x

Then filter it:

results = (f(x) for x in interesting_values) # A generator expression; almost a list comptehension

valid_results = filter(lambda x: x is not None, results)

This is a stripped down version of what's often referred to as the "Optional Pattern". Return a special sentinal value on error (None in this case), else, return a valid value. Normally the Optional type is a special type and the sentinal value is a subclass of that type (or something similar), but that's not necessary here.

Answer 2

I'm going to assume here that you have no control over the source of f. If you do, the first suggestion is to simply rewrite f not to throw exceptions, as it's clear that you are expecting that execution path to occur, which by definition makes it not exceptional. However, if you don't have control over it, read on.

If you have a function that might fail and want its "failure" to be ignored, you can always just wrap the function

def safe_f(x):
  try:
    return f(x)
  except ValueError:
    return None

result = filter(lambda x: x is not None, map(safe_f, values))

Of course, if f could return None in some situation, you'll have to use a different sentinel value. If all else fails, you could always go the route of defining your own _sentinel = object() and comparing against it.

Answer 3

You could add another layer on top of your function. A decorator if you will, to transform the exception into something more usable. Actually this is a function that returns a decorator, so two additional layers:

from functools import wraps

def transform(sentinel=None, err_type=ValueError):
    def decorator(f):
        @wraps(f)
        def func(*args, **kwargs):
            try:
                return f(*args, **kwargs)
            except err_type:
                return sentinel
        return func
    return decorator

@transform()
def f(...): ...

interesting = range(10)
result = [y for y in (f(x) for x in interesting) if y is not None]

This solution is tailored for the case where you get f from somewhere else. You can adjust transform to return a decorator for a given set of exceptions, and a sentinel value other than None, in case that's a valid return value. For example, if you import f, and it can raise TypeError in addition to ValueError, it would look like this:

from mystuff import f, interesting

sentinel = object()
f = transform(sentinel, (ValueError, TypeError))(f)
result = [y for y in (f(x) for x in interesting) if y is not sentinel]

You could also use the functional version of the comprehension elements:

result = list(filter(sentinel.__ne__, map(f, interesting)))