POST request with Multipart/form-data. Content-type not correct

Question

We're trying to write a script with python (using python-requests a.t.m.) to do a POST request to a site where the content has to be MultipartFormData. When we do this POST request manually (by filling in the form on the site and post), using wireshark, this came up (short version):

Content-Type: multipart/form-data;
Content-Disposition: form-data; name="name"
Data (8 Bytes)
    John Doe

When we try to use the python-requests library for achieving the same result, this is sent:

Content-Type: application/x-pandoplugin
Content-Disposition: form-data; name="name"; filename="name"\r\n
Media type: application/x-pandoplugin (12 Bytes)
    //and then in this piece is what we posted://
    John Doe

The weird thing is that the 'general type' of the packet indeed is multipart/form-data, but the individual item sent (key = 'name', value= 'John Doe') has type application/x-pandoplugin (a random application on my pc I guess).

This is the code used:

response = s.post('http://url.com', files={'name': 'John Doe'})

Is there a way to specify the content-type of the individual items instead of using the headers argument (which only changes the type of the 'whole' packet)?

We think the server doesn't respond correctly due to the fact that it can't understand the content-type we send it.

Little update: I think the different parts of the multipart content are now identical to the ones sent if I do the POST in the browser, so that's good. Still the server doesn't actually do the changes I send it with the script. The only thing that still is different is the order of the different parts.

For example this is what my browser sends:

Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part:  (text/plain)
    Content-Disposition: form-data; name="file"; filename="ex.txt"\r\n
    Content-Type: text/plain\r\n\r\n
    Line-based text data: text/plain
        lore ipsum blabbla

Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part: 
    Content-Disposition: form-data; name="seq"\r\n\r\n
    Data (2 bytes)

Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part: 
    Content-Disposition: form-data; name="name"\r\n\r\n
    Data (2 bytes)

And this is what the script (using python-requests) sends:

Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part: 
    Content-Disposition: form-data; name="name"\r\n\r\n
    Data (2 bytes)

Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part:  (text/plain)
    Content-Disposition: form-data; name="file"; filename="ex.txt"\r\n
    Content-Type: text/plain\r\n\r\n
    Line-based text data: text/plain
        lore ipsum blabbla

Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part: 
    Content-Disposition: form-data; name="seq"\r\n\r\n
    Data (2 bytes)

Could it be possible that the server counts on the order of the parts? According to Multipart upload form: Is order guaranteed?, it apparently is? And if so, is it possible to explicitly force an order using the requests library? And to make things worse in that case: There is a mixture of a file and just text values.

So forcing an order seems rather difficult. This is the current way I do it:

s.post('http://www.url.com', files=files,data = form_values)

EDIT2: I did a modification in the requests plugin to make sure the order of the parts is the same as in the original request. This doesn't fix the problem so I guess there is no straightforward solution for my problem. I'll send a mail to the devs of the site and hope they can help me!

Answer 1

your code looks correct.

requests.post('http://url.com', files={'name': 'John Doe'})

... and should send a 'multipart/form-data' Post.

and indeed, I get something like this posted:

Accept-Encoding: gzip, deflate, compress
Connection: close
Accept: */*
Content-Length: 188
Content-Type: multipart/form-data; boundary=032a1ab685934650abbe059cb45d6ff3
User-Agent: python-requests/1.2.3 CPython/2.7.4 Linux/3.8.0-27-generic

--032a1ab685934650abbe059cb45d6ff3
Content-Disposition: form-data; name="name"; filename="name"
Content-Type: application/octet-stream

John Doe
--032a1ab685934650abbe059cb45d6ff3--

I have no idea why you'd get that weird Content-Type header:

Content-Type: application/x-pandoplugin

I would begin by removing Pando Web Plugin from your machine completely, and then try your python-requests code again. (or try from a different machine)

Answer 2

As of today you can do:

response = s.post('http://url.com', files={'name': (filename, contents, content_type)})