How to serve binary data from AWS API Gateway with proxy integration?

Question

I am building a serverless web site with AWS API Gateway (APIG) and Lambda functions. I must use proxy integration because the Lambda function behind has to receive http request headers. At the same time, some binary data need to be served. In my case, the favicon.ico file. Other people might want to serve dynamically created PDF or Excel files. APIG has binary support for this purpose. Encode data with base64 and configure that MIME content type so it can get decoded before it gets served to client. However, this doesn't work with proxy integration. Proxy integration just skips the integration response part.

I tried to redirect the request of favicon.ico to S3 endpoint, but browsers show strange behaviors. Because the ico file is from a different and redirected domain, not the same domain.

Encoding it with base64 and letting the client browser decode it is not an option since this isn't standard and might not work in all browsers.

I assume I can't do anything else until AWS adds a new feature for this. Has anyone looked into this problem? Any idea or suggestion?

Answer 1

Answering my own question here. You should go to AWS forums for AWS questions. Not mamy AWS users here.

The answer: Binary support with proxy integration DOES WORK and it works for both incoming and outgoing response.

There are 3 factors related to this:

1. the MIME types in binary support settings for APIG (I set this on console)
2. the "isBase64Encoded" value in both incoming and outgoing JSON
3. the "Content-type" value in outgoing JSON

The answer above means that yes, you can accept files and yes again, you can spit out files with proxy integration.

When user 'posts' with body and if the MIME type you set matches it, APIG encodes the whole body part in base64 and indicates this with "isBase64Encoded" value. The same thing happens with outgoing responses too. If you want to response with binary data, you encode it in base64 and set that value true in outgoing JSON.

For incoming requests, it depends on only one factor, the MIME type you set. For outgoing, the both conditions, indicator in JSON and MIME type must match.

For simplicity, I just set the MIME type to */*. Whenever user submits anything in the body, let APIG encode then I decode. Whenever I response with binary, I just set the indicator and encode in base64. I do not do this with other types such as test/html (not compressed).

Answer 2

In case of Lambda proxy integration, there are two requirements:

• The binary should be base64-encoded on the function's response, include the 'isBase64Encoded' attribute set to 'true' and a 'Content-Type' header to indicate to the client the type of data that it returns. See example in [1].

• The 'binaryMediaTypes' on the API settings must support such 'Content-Type' so the payload could be handle as binary [2].

Note: For proxy integrations, API Gateway passes the entire request through to your backend, and you do not have the option to modify the passthrough behaviors [3].

If you use '/' wildcard as the binary media type and the response from the Lambda is flagged as not base64 encoded, then the Lambda response will not be treated as binary, and the response will be returned in text format ( e.g. base64 encoded in case you encoded the file). Please note if your client is not prepared to handle the text response, it may see your binary as corrupted..

API Gateway applies a base64 decode for mime-type registered for binary support, if the payload received by the backend integration is base64 encoded.

Returning binary with a mapping template(in general):

The issue with returning binary with a mapping template if that if you need to select "CONVERT_TO_BINARY" in the Integration response, this will be applied before the mapping template is applied and this will result in the failure. Mapping template are meant to be used only and exclusively for application/json payloads. Using it with other content types and binary does not work well as mapping templates were not meant to work with those.

The mapping template you mentioned will not work. If you want to have a binary file as response, the CONVERT_TO_BINARY will be applied before the mapping template, hence this will not work. This is a limitation of mapping templates and hence it would be recommend to use Lambda Proxy Integration, or retuning from your Lambda template only the base64 encoded payload, no JSON or other data in it. Using $util.base64Decode will also not result in a valid binary file as API Gateway would still try to interpret it as application/json and hence the payload will be corrupted.

Following are the options you can consider:

1- Return the application/pdf as base64 encoded string from your lambda and then apply CONVERT_TO_BINARY in the Integration Response Passthrough. You will also need to statically specify the Content-Type response header in the integration response and method responses. Leave the response mapping template empty so the body is passed through and converted to a proper binary file.

2- Use Lambda proxy integration and return the payload as you were doing, with the headers, body as base64 encoded payload and 'isBase64Encoded': true. API Gateway will take care of setting the proper headers and return the file as binary.