How to post Multipart Form Data through python aiohttp ClientSession

Question

I am trying to asynchronously send some Multipart-Encoded Form Data as a post request, mainly a file and two other fields.

Before trying to use asyncio I was doing the process synchronously with requests-toolbelt MultipartEncoder (https://github.com/requests/toolbelt) which worked great for normal requests, but did not work when using aiohttp for async. aiohttp provides 2 multipart classes, a FormData() class and a MultipartWriter() class, neither of which have given me much success.

After some testing, it seems like the difference is that when I use the toolbelt MultipartEncoder() the request sends the data in the form section of the post request as it should. However, when using aiohttp the request is put into the body section of the request. Not sure why they are acting differently

def multipartencode() -> ClientResponse():
        # Using MultipartEncoder
        m = MultipartEncoder(
            fields={'type': type_str,
                    'metadata': json.dumps(metadata),
                    'file': (filename, file, 'application/json')}
        )

        # Using FormData
        data = FormData()
        data.add_field('file', file, filename=filename,
                       content_type='multipart/form-data')
        data.add_field('type', type_str, content_type='multipart/form-data')
        data.add_field('metadata', json.dumps(metadata),
                       content_type='multipart/form-data')

        # Using MultipartWriter
        with MultipartWriter('multipart/form-data') as mpwriter:
            part = mpwriter.append(
                file, {'CONTENT-TYPE': 'multipart/form-data'})
            part.set_content_disposition('form-data')
            part = mpwriter.append_form([('type', type_str)])
            part.set_content_disposition('form-data')
            part = mpwriter.append_form([('metadata', json.dumps(metadata))])
            part.set_content_disposition('form-data')


        # send request with ClientSession()
        resp = await session.post(url=url, data=data, headers=headers)
        return resp

How can I properly format/build the multipart-encoded request to get it to send using aiohttp?

Answer 1

I was struggling with this for hours. My particular case was to send an email with file attachment to mailgun. Same thing that was addressed by the comment above. Please find below the working code:

import asyncio

import aiohttp


async def send():
    url = "<<mailgun_url>>"
    api_key = "<<mailgun_api_key>>"

    mail_gun_data = {
        "from": "<<from>>",
        "to": "<<to>>",
        "subject": "Subject",
        "text": "Testing Mailgun",
        "attachment": open("<<file_path>>", "rb")
    }

    async with aiohttp.ClientSession() as session:
        with aiohttp.MultipartWriter("form-data") as mp:
            for key, value in mail_gun_data.items():
                part = mp.append(value)
                part.set_content_disposition('form-data', name=key)
            resp = await session.post(
                url,
                auth=aiohttp.BasicAuth("api", api_key),
                data=mp,
            )


if __name__ == '__main__':
    asyncio.run(
        send()
    )

I hope this is helpful and will save someone's time.