Python - HTTP multipart/form-data POST request

Question

I would like to upload a file to a web server. From what I have read, the best way to do this is to use the multipart/form-data encoding type on an HTTP POST request.

My research seems to indicate that there is no simple way to do this using the Python standard library. I am using Python 3.

(Note: see a package called requests (PyPI Link) to easily accomplish this)

I am currently using this method:

import mimetypes, http.client
boundary = 'wL36Yn8afVp8Ag7AmP8qZ0SA4n1v9T' # Randomly generated
for fileName in fileList:
    # Add boundary and header
    dataList.append('--' + boundary)
    dataList.append('Content-Disposition: form-data; name={0}; filename={0}'.format(fileName))

    fileType = mimetypes.guess_type(fileName)[0] or 'application/octet-stream'
    dataList.append('Content-Type: {}'.format(fileType))
    dataList.append('')

    with open(fileName) as f: 
        # Bad for large files
        dataList.append(f.read())

dataList.append('--'+boundary+'--')
dataList.append('')
contentType = 'multipart/form-data; boundary={}'.format(boundary)

body = '\r\n'.join(dataList)
headers = {'Content-type': contentType}

conn = http.client.HTTPConnection('http://...')
req = conn.request('POST', '/test/', body, headers)

print(conn.getresponse().read())

This works to send text.

There are two issues: This is text only, and the whole text file must be stored in memory as a giant string.

How can I upload any binary file? Is there a way to do this without reading the whole file into memory?

Answer 1

Take a look at small Doug Hellmann's urllib2, translated by me to python3.

I use it nearly this way:

import urllib.request
import urllib.parse
from lib.multipart_sender import MultiPartForm

myfile = open('path/to/file', 'rb')
form = MultiPartForm()
form.add_field('token', apipost[mycgi['domain']]._token)
form.add_field('domain', mycgi['domain'])
form.add_file('file', 'logo.jpg', fileHandle=myfile)
form.make_result()

url = 'http://myurl'
req1 = urllib.request.Request(url)
req1.add_header('Content-type', form.get_content_type())
req1.add_header('Content-length', len(form.form_data))
req1.add_data(form.form_data)
fp = urllib.request.urlopen(req1)
print(fp.read()) # to view status