Menu
I want to make a post request to a HTTPS-site that should respond with a .csv file. I have this Python code:
url = 'https://www.site.com/servlet/datadownload'
values = {
'val1' : '123',
'val2' : 'abc',
'val3' : '1b3',
}
data = urllib.urlencode(values)
req = urllib2.Request(url,data)
response = urllib2.urlopen(req)
myfile = open('file.csv', 'wb')
shutil.copyfileobj(response.fp, myfile)
myfile.close()
But 'm getting the error:
BadStatusLine: '' (in httplib.py)
I've tried the post request with the Chrome Extension: Advanced REST client (screenshot) and that works fine.
What could be the problem and how could I solve it? (is it becasue of the HTTPS?)
EDIT, refactored code:
try:
#conn = httplib.HTTPSConnection(host="www.site.com", port=443)
=> Gives an BadStatusLine: '' error
conn = httplib.HTTPConnection("www.site.com");
params = urllib.urlencode({'val1':'123','val2':'abc','val3':'1b3'})
conn.request("POST", "/nps/servlet/exportdatadownload", params)
content = conn.getresponse()
print content.reason, content.status
print content.read()
conn.close()
except:
import sys
print sys.exc_info()[:2]
Output:
Found 302
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>302 Found</TITLE>
</HEAD><BODY>
<H1>Found</H1>
The document has moved <A HREF="https://www.site.com/nps/servlet/exportdatadownload">here</A>.<P>
<HR>
<ADDRESS>Oracle-Application-Server-10g/10.1.3.5.0 Oracle-HTTP-Server Server at mp-www1.mrco.be Port 7778</ADDRESS>
</BODY></HTML>
What am I doing wrong?
320
Is there a reason you've got to use urllib? Requests is simpler, better in almost every way, and abstracts away some of the cruft that makes urllib hard to work with.
As an example, I'd rework you example as something like:
import requests
resp = requests.post(url, data=values, allow_redirects=True)
At this point, the response from the server is available in resp.text, and you can do what you'd like with it. If requests wasn't able to POST properly (because you need a custom SSL certificate, for example), it should give you a nice error message that tells you why.
Even if you can't do this in your production environment, do this in a local shell to see what error messages you get from requests, and use that to debug urllib.
110
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
