AWS Lambda Python - Return BytesIO file?

Question

I'm setting up a function in AWS Lambda using python 3.7 and it won't let me return a bytes type

Please notice that this is not an issue with API Gateway, I'm invoking the lambda directly.

The error is : Runtime.MarshalError, ... is not JSON serializable

output = BytesIO()
#Code that puts an excel file into output...
return {
        'Content-Disposition': 'attachment; filename="export.xlsx"',
        'Content-Type': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
        'body' : output.getvalue()
    }

If I do :

'body' : str(output.getvalue())

It outputs a corrupted file because it adds b'' to the string

If I do :

'body' : base64.b64encode(output.getvalue()).decode()

It also outputs a corrupted file, probably because it changes the binary representation of the file.

Maybe I need to upload to S3? But it doesn't fit in my flow, this is a one time file creation and it would stay in "S3 Limbo" until TTL

Answer 1

It is not possible to return unencoded binary data from a direct invoked AWS Lambda function.

Per the docs:

If the handler returns objects that can't be serialized by json.dumps, the runtime returns an error.

The reason you can do this with API Gateway is because API Gateway is performing the conversion of the base64 JSON content your function returns into binary for you. (See documentation here)

I would need to know more about how you are invoking Lambda to be sure but I suspect you could implement this same base64 decode logic into your direct invoke client. Alternatively, if you wanted to keep the client as simple as possible, use S3 with a lifecycle hook to keep the bucket from filling up with temporary files.