Handling Failure in Twisted

Question

This is my simple HTTP client for some api:

# -*- coding: utf-8 -*-
import settings
from twisted.internet import reactor
from twisted.web.client import Agent
from twisted.web.http_headers import Headers

params = {
    'url': 'http://api.vk.com/api.php',
    'id':260,
    }


def params_for_get():
    return '&'.join(["%s=%s" % (key,val) for key, val in params.items()])


agent = Agent(reactor)
d = agent.request(
    'GET',
    "%s?%s" % (settings.APPLICATION_URL, params_for_get()),
    Headers({'User-Agent': ['Twisted Web Client Example'],
             'Content-Type': ['text/x-greeting']}),
    '')

def cbResponse(*args, **kwargs):
    print args, kwargs
    print 'Response received'


def cbShutdown(ignored):
    reactor.stop()

def cbError(failure):
    print type(failure.value), failure # catch error here


d.addCallbacks(cbResponse, cbError)
d.addBoth(cbShutdown)


reactor.run()

When I start program, I catch error:

<class 'twisted.web._newclient.RequestGenerationFailed'> [Failure instance: Traceback (failure with no frames): <class 'twisted.web._newclient.RequestGenerationFailed'>: [<twisted.python.failure.Failure <type 'exceptions.AttributeError'>>]
]

But I don't know, where this error happened. How I can know it? I tried to display traceback for

<twisted.python.failure.Failure <type 'exceptions.AttributeError'>>

but I could not get.

Answer 1

That Failure instance is wrapping another Failure instance, and isn't printing much information about the one inside. This awkwardness is Twisted's fault; the twisted.web._newclient._WrapperException class stores a reasons attribute, but doesn't seem to care about printing information about those reasons in its __str__ method.

You could see the rest of your problem if you add another line to your cbError() function:

failure.value.reasons[0].printTraceback()

I can reproduce the error here, and with the extra information it's apparent that your fourth parameter to Agent.request() is supposed to be an IBodyProducer provider, but you're passing an empty string instead. Try removing that last parameter.

Answer 2

The answer by the-paul above is correct. I wanted to provide this short function that I found useful in flattening out the Failures, but I couldn't put it in a comment:

def unwrap_failures(err):
    """
    Takes nested failures and flattens the nodes into a list.
    The branches are discarded.
    """
    errs = []
    check_unwrap = [err]
    while len(check_unwrap) > 0:
        err = check_unwrap.pop()
        if hasattr(err.value, 'reasons'):
            errs.extend(err.value.reasons)
            check_unwrap.extend(err.value.reasons)
        else:
            errs.append(err)
    return errs