Pythonic way to combine `for` and `try` blocks

Question

I have code where I am parsing a JSON feed.

For each array I have code that looks like:

for node in parse_me:
    # It's important that one iteration failing doesn't cause all iterations to fail.
    try:
        i = node['id'] # KeyError?
        function_that_needs_int (i) # TypeError?
        # possibly other stuff

    except Exception as e:
        LogErrorMessage ('blah blah blah {} in node {}'.fmt(e, node))

I do not like that this makes my for loops double-nested just because I need to stop exceptions from aborting the loop. Is there a way to flatten this code?

Answer 1

Then you should do things like

def iterate_safe(parse_me, message, action):
    for node in parse_me:
        try:
            action(node)
        except Exception as e:
            LogErrorMessage(message.fmt(e, node))

and then call it like

def action(node):
    do_whatever_must_be_done_with(node)

iterate_safe(parse_me, action, 'blah blah blah {} in node {}')
iterate_safe(parse_me, other_action, 'spam ham {} in node {}')

Answer 2

EDIT: The original question seemed to imply that the entire parse operation was in one giant for-loop; my answer has been modified to reflect comments below.

Instead of writing multiple for-loops, each of which must include a try/catch block, write functions describing what must be done within the loops, and write a decorator to apply to them that will surround each one with the for-loop and the try/catch logging logic. This is a bit like glglgl's solution, but a bit more Pythonic (in my opinion). For example:

def apply_to_nodes_and_log_errs(node_visit_func):
    def safe_iterating_visitor(nodes_to_parse):
        for node in nodes_to_parse:
            try:
                node_visit_func(node)
            except StandardError as e:
                LogErrorMessage ('blah blah blah {} in node {}'.fmt(e, node))
    return safe_iterating_visitor

@apply_to_nodes_and_log_errs
def action_one(node):
    # ... "lots of stuff" :D

@apply_to_nodes_and_log_errs
def action_two(node):
    # different stuff

If you'd rather break the decorator into chunks:

def iterate_over_nodelist(node_visit_func):
    def iterating_visitor(nodes_to_parse):
        for node in nodes_to_parse:
            node_visit_func(node)
    return iterating_visitor

def safely_visit_log_errs(node_visit_func):
    def safe_logging_visitor(node_to_visit):
        try:
            node_visit_func(node)
        except StandardError as e:
            LogErrorMessage ('blah blah blah {} in node {}'.fmt(e, node))
    return safe_logging_visitor

def apply_to_nodes_and_log_errs(node_visit_func):
    return iterate_over_nodelist(safely_visit_log_errs(node_visit_func))

# ... write visit functions

This could be further improved using functools.wraps.

Note that although this may look a little ugly if your standard is "use as few levels of indentation as possible," it's actually quite Pythonic; there's really no way to avoid quite a few indentation levels when writing decorators.

Finally, note that change from Exception to StandardError, which I still strongly recommend.