converting curl call to python requests

Question

I've got a problem converting curl request into Python requests call:

here is how curl call looks like: (taken from: http://developers.payu.com/en/restapi.html#creating_new_order_api )

curl -X POST https://secure.payu.com/api/v2_1/orders \
-H "Content-Type: application/json" \
-H "Authorization: Bearer 3e5cac39-7e38-4139-8fd6-30adc06a61bd" \
-d '{
    "notifyUrl": "https://your.eshop.com/notify",
    "customerIp": "127.0.0.1",
    "merchantPosId": "145227",
    "description": "RTV market",
    "currencyCode": "PLN",
    "totalAmount": "21000",
    "products": [
        {
            "name": "Wireless Mouse for Laptop",
            "unitPrice": "15000",
            "quantity": "1"
        },
        {
            "name": "HDMI cable",
            "unitPrice": "6000",
            "quantity": "1"
        }
    ]
}'

and here is what I wrote in requests:

import json
import requests
headers = {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer 3e5cac39-7e38-4139-8fd6-30adc06a61bd',
}

data = {
    "notifyUrl": "https://your.eshop.com/notify",
    "customerIp": "127.0.0.1",
    "merchantPosId": "145227",
    "description": "RTV market",
    "currencyCode": "PLN",
    "totalAmount": "21000",
    "products": [
        {
            "name": "Wireless Mouse for Laptop",
            "unitPrice": "15000",
            "quantity": "1"
        },
        {
             "name": "HDMI cable",
             "unitPrice": "6000",
             "quantity": "1"
        }
    ]
}

resp2 = requests.post('https://secure.payu.com/api/v2_1/orders', headers=headers, json=data)
print(resp2.json())

curl as a response prints out:

{"orderId":"V6GRPMNRLR160429GUEST000P01","status":{"statusCode":"SUCCESS"},"redirectUri":"https://secure.payu.com/pl/standard/co/summary?sessionId=HtnLqVtBJ5tcOKG2nX03TKwAXOYtXPHe&merchantPosId=145227&timeStamp=1461948331350&showLoginDialog=false&apiToken=9f31fcd1d0d1c5fde8aa57c2b16b5d6bbdfe81543a5f6a12cd39955a487fdaab"}

whereas python requests:

/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.        
  SNIMissingWarning                                                                                                                                                                                                       
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.                                                                           
  InsecurePlatformWarning                                                                                                                                                                                                 
Traceback (most recent call last):                                                                                                                                                                                        
  File "/home/ubuntu/workspace/src/billing/testapi.py", line 30, in <module>                                                                                                                                              
    print(resp2.json())                                                                                                                                                                                                   
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 808, in json                                                                                                                                     
    return complexjson.loads(self.text, **kwargs)                                                                                                                                                                         
  File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 488, in loads                                                                                                                                      
    return _default_decoder.decode(s)                                                                                                                                                                                     
  File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 370, in decode                                                                                                                                      
    obj, end = self.raw_decode(s)                                                                                                                                                                                         
  File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 389, in raw_decode                                                                                                                                  
    return self.scan_once(s, idx=_w(s, idx).end())                                                                                                                                                                        
simplejson.scanner.JSONDecodeError: Expecting value: line 8 column 1 (char 7)   

does anyone know why it's not working with requests ? thanks

Answer 1

The site responds with a 302 status code, including a Location header:

>>> resp = requests.post('https://secure.payu.com/api/v2_1/orders', headers=headers, json=data)
>>> resp.history
[<Response [302]>, <Response [302]>]
>>> resp.history[0].headers['location']
'https://secure.payu.com/pl/standard/co/summary?sessionId=rrQ97fR2bxQJhUqCDORCbRa0uA4WlxJi&merchantPosId=145227&timeStamp=1461950205252&showLoginDialog=false&apiToken=f3599eca78ad55a16d84608e69f3ac7458b782598b064b337be07e8bd6c2c1d5'
>>> resp.history[0].text
u'{"orderId":"Z9W2H77TVD160429GUEST000P01","status":{"statusCode":"SUCCESS"},"redirectUri":"https://secure.payu.com/pl/standard/co/summary?sessionId=rrQ97fR2bxQJhUqCDORCbRa0uA4WlxJi&merchantPosId=145227&timeStamp=1461950205252&showLoginDialog=false&apiToken=f3599eca78ad55a16d84608e69f3ac7458b782598b064b337be07e8bd6c2c1d5"}'

You can look at the redirection history (like I did above) by looking at the response.history list, which contains preceding requests. resp.history[0] was the initial response here. See Redirection and History in the documentation.

Note that the data is right there on the initial response.

The site does this when you use any Accept-Encoding header; you get the same result if you add

-H "Accept-Encoding: gzip, deflate"

to the curl command, or by adding the same header (with any value) to the supplied payu.apiari.io console (click Headers, then add the Accept-Encoding header and add a value for the header).

You should consider this a bug in the site, as specifying that header is a perfectly normal thing to do. In fact, the Python httplib library (used indirectly by requests) sets a default value for that header if you omit it yourself, so this is not something requests can even switch off. As such, I've reported this to PayU as a bug.

As a work-around, you can use the response.history[0] reference, or better yet, tell requests not to follow the redirect in the first place:

>>> resp = requests.post('https://secure.payu.com/api/v2_1/orders', headers=headers, json=data, allow_redirects=False)
>>> resp.json()
{u'orderId': u'NBWTP5WNKK160429GUEST000P01', u'status': {u'statusCode': u'SUCCESS'}, u'redirectUri': u'https://secure.payu.com/pl/standard/co/summary?sessionId=PcvceJHf5En60Dier5gKxCyExiva4qh0&merchantPosId=145227&timeStamp=1461950329704&showLoginDialog=false&apiToken=5962ef901010ca4f8ef6491619217c060f4d53ed0d8e4eadb513d4e0811fc992'}

But take care that ignoring the 302 may interfere with the normal status codes, I see that a 302 may indicate that additional security info is required.