How can I bypass the 10MB limit of AWS API gateway and POST large files to AWS lambda?

Question

what I want

An API which takes file and some parameters using POST and gives back a JSON response.

curl -X POST www.endpoint.com \
   -F file=@/myfile.txt \
   -F foo=bar # other params

I have this working with Lambda + API gateway using binary data but 10MB limit is the issue.

I have considered a POST API which uploads file to S3. The event generated is then read by Lambda. But for this I have few questions-

1. Where will my other parameters go?
2. How will Lambda return back the response?

Answer 1

Your use case simply isn't possible with one API call if you want to stick with a serverless solution.

A possible serverless solution would be a 3 step process for the client.

Step 1

Call api1 to get a signed url for S3. This would point to a Lambda that creates a UUID and uses that UUID to construct a signed URL for S3 (i.e. uses the UUID as the filename of the file being received). The response would be the URL and the UUID.

Step 2

PUT file to s3 using the signed URL.

Step 3

Call api2 and pass the UUID and what ever other parameters are required. This api also points to a Lambda which now knows where the file is (thanks to the UUID) and has whatever other parameters are required to process the file and give a response.