Python Requests - Post a zip file with multipart/form-data

Question

I'm playing around with the Python Requests module that has so far been a delight.

However, I've run into an issue whilst attempting to post a zip file using multipart/form-data.

I'm using Digest authentication and have been able to successfully post other file types e.g. .xls etc.

I'm creating a post request using:

file = open('/Users/.../test.zip', 'rb').read()
r = requests.post(url, auth=HTTPDigestAuth('dev', 'dev'), data = {"mysubmit":"Go"}, files={"archive": ("test.zip", file)})

This errors out and gives:

requests.exceptions.ConnectionError: HTTPConnectionPool(host='10.2.2.70', port=80): Max retries exceeded with url: /plugin_install 
(Caused by <class 'socket.error'>: [Errno 32] Broken pipe)

I've tried with smaller size zip files and changing the data/files values, and the same error occurs.

Am I missing something obvious?

Thanks for any light you can shed!

Answer 1

As far as requests is concerned, there is no difference between a zip file and any other binary blob of data.

Your server is broken here; it is cutting of the connection when you send it a zip file. That is not something requests can do anything about.

You may want to test against http://httpbin.org/ when you run into problems like these; it is a testing service built by the author of the requests library.

Another tip: you don't need to read the whole file object into memory when sending. Just pass the object itself to requests instead:

fileobj = open('/Users/.../test.zip', 'rb')
r = requests.post(url, auth=HTTPDigestAuth('dev', 'dev'), data = {"mysubmit":"Go"}, files={"archive": ("test.zip", fileobj)})

Demo against httpbin.org:

>>> import requests
>>> fileobj = open('/tmp/test.zip', 'rb')
>>> r = requests.post('http://httpbin.org/post', data={"mysubmit":"Go"}, files={"archive": ("test.zip", fileobj)})
>>> r
<Response [200]>
>>> r.json()
{u'origin': u'217.32.203.188', u'files': {u'archive': u'data:application/zip;base64,<long base64 body omitted>'}, u'form': {u'mysubmit': u'Go'}, u'url': u'http://httpbin.org/post', u'args': {}, u'headers': {u'Content-Length': u'57008', u'Accept-Encoding': u'gzip, deflate, compress', u'Connection': u'close', u'Accept': u'*/*', u'User-Agent': u'python-requests/1.2.3 CPython/2.7.5 Darwin/12.4.0', u'Host': u'httpbin.org', u'Content-Type': u'multipart/form-data; boundary=9aec1d03a1794177a38b48416dd4c811'}, u'json': None, u'data': u''}