Correct way to make a Python HTTPS request using requests module?

Question

I have to make an HTTPS request in Python, and I am using the requests module to try to make my life easier.

The request needs to have a header and 3 FORM parameters URL encoded. This is what I am doing:

header = {'Content-type': 'application/x-www-form-urlencoded', 'Authorization':'Basic ' + encoded_string, 'Connection': 'Keep-Alive', 'Host':'host.host.com'}

payload='grant_type=authorization_code&code=' + request.args['code'] + '&state=' + request.args['state'] + '&redirect_uri=http://xxx.xyz.com/request_listener'

url = 'https://serviceprovider.xxx.com/auth/j_oauth_resolve_access_code'

response = requests.post(url, data=payload, headers=header, verify=False)

When I try to return the content or text of the response, I get an empty string. However, when I print the actual response object, it says it is a <Response [200]>, but if this were actually a 200 OK then the server I am POSTing too should go to the redirect_uri I have specified and I would get a notification there.

This is not happening, and I am mystified as to why.

Answer 1

Your code is fighting the Requests library: you're doing a lot of stuff yourself that Requests will do for you.

Firstly, don't form-encode your data yourself, let Requests do it by providing a dictionary to data, like @flyer's answer suggested.

When you do this, Requests will also correctly set the Content-Type header, so you don't have to. Also, please don't send a Connection header: Requests will manage it for you. The same applies to Host headers: sending a Host header can only cause problems.

Finally, don't set the Authorization header yourself, let Requests do it by providing it with your authentication credentials. The idiomatic Requests code would be:

payload = {
    'grant_type': 'authorization_code', 
    'code': request.args['code'],
    'state': request.args['state'],
    'redirect_uri': 'http://xxx.xyz.com/request_listener',
}

url = 'https://serviceprovider.xxx.com/auth/j_oauth_resolve_access_code'

response = requests.post(url, data=payload, verify=False)

If that doesn't work, then I would suspect your payload data is bad.

Answer 2

It seems that there are two mistakes.

The first one:
When you want to post data, the data format should be like this:

payload = {
    'grant_type': 'authorization_code', 
    'code': request.args['code'],
    'state': request.args['state'],
    'redirect_uri': 'http://xxx.xyz.com/request_listener',
}

The second:
Requests could verify SSL certificates for https requests automatically and it sets verify=True as default. You set verify=False and that means that you want to ignore the ssl verification. That maybe not what you want. Here is the doc