How to handle OpenSSL.SSL.Error while using twisted.web.client.Agent on Facebook graph api?

Question

I am running the ff. code from a virtualenv on Mac OS X (Yosemite):

    # testfb.py
    from twisted.internet import reactor
    from twisted.python import log
    from twisted.web.client import Agent

    GRAPH_API = "https://graph.facebook.com/v2.5"

    def stop(_):
        reactor.stop()

    def get_me(access_token):
        agent = Agent(reactor)
        uri = "{}/me?access_token={}".format(GRAPH_API, access_token)
        log.msg("uri:" + uri)
        return agent.request("GET", uri)

    if __name__ == "__main__":
        import sys
        access_token = sys.argv[1]
        d = get_me(access_token)
        d.addErrback(log.err)
        d.addCallback(stop)
        reactor.run()

And I get:

    Failure: twisted.web._newclient.ResponseNeverReceived: [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]>]

I don't have this issue when I call the uri in curl.

BTW, I also installed service_identity using pip on the virtualenv.

Answer 1

You probably don't have any OpenSSL trust roots configured. This used to happen automatically by accident because Cryptography linked against the built-in version of OpenSSL from OS X. Since the headers for that version were removed in El Capitan, Cryptography now ships wheels with their own built-in version of pyOpenSSL.

This is a bug in Twisted, and we know about it; you can read more about it here: https://twistedmatrix.com/trac/ticket/6372. It's been open for a while; because it worked by accident for a long time, it hasn't been a high priority. Thanks to bug reports from folks like you, this is changing...

In the meanwhile, though, you have two possible options.

One is that you can install OpenSSL from Homebrew, which will automatically put some certificate authority certificates in the default location that Cryptography's OpenSSL is already looking for them, with brew install openssl. (You can see these certificate authority certificates in /usr/local/etc/openssl/cert.pem after installing it)

Another is that you can get some certificates from Certifi, and then set the (undocumented) environment variable that tells OpenSSL to look for them, by doing, for example, export SSL_CERT_FILE="$(python -m certifi)" in your shell before invoking your Python program.

Sorry for the, and I hope this answers your question!