At the moment I think I have a foundational misunderstanding of how to extract/pass data from lambda functions & I'm seeking clarification
Example: Let's say I want to pass some data to a lambda function ie {"hello":"world"}
and do so by passing data to that Lambda function by creating a REST endpoint in AWS API Gateway
At the moment I understand there are three ways to extract data:
1) event.queryStringParameters (makes sense)
ex. We can attach query parameters to the request URL: https://fakefakefake.execute-api.us-west-2.amazonaws.com/test/myapi?hello=world and in the lambda function:
const data = event.queryStringParameters.hello; // 'world'
2) event.body (makes sense & this is possible because of "Lambda Proxy Integration")
ex. If we attach the data in the body of the POST/PUT/etc request using Lambda Proxy Integration (ie forward all the data), we can access it via event.body
& within the lambda function (but making sure to JSON.parse the event.body since Lambda Proxy Integration will pass through stringified JSON & not valid/"real" JSON):
const parsedBody = JSON.parse(event.body); // should wrap in try/catch
const data = parsedBody.hello; // 'world'
3) Directly on the event object (Unclear)
ex. This case is unclear at the moment-- pass data to the lambda function from a REST endpoint setup in API Gateway where it is then accessible directly from the event object?
const data = event.hello; // 'world'
What is an example of how to pass data "directly" on the event object in a Lambda function like in case #3? I THINK this case requires that I create a "mapping template" when setting up the API/Lambda but I'm still unclear.
For a simple Node script, case 2 appears to have the "overhead" of parsing event body from stringified JSON so that's an understandable downside, but in addition to how to do it why or when would Case 3 be a more desirable approach?
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html
Lambda proxy integration: Getting json body in aws Lambda via API gateway
Lambda proxy integration (AWS example): https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html
Lambda Proxy Integration vs Lambda Proxy (option #2 vs option #3 above): Lambda Integration vs. Lambda Proxy: Pros and Cons
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html
You can create multiple event source mappings to process the same data with multiple Lambda functions, or to process items from multiple data streams with a single function.
Lambda Functions can be triggered in different ways: an HTTP request, a new document upload to S3, a scheduled Job, an AWS Kinesis data stream, or a notification from AWS Simple Notification Service (SNS).
Example EventBridge (CloudWatch Events) eventYou can also create a Lambda function and direct AWS Lambda to invoke it on a regular schedule. You can specify a fixed rate (for example, invoke a Lambda function every hour or 15 minutes), or you can specify a Cron expression.
Lambda evaluates every request through an internal Application Load Balancer to manage and balance the requests against the underlying infrastructure, the load balancer forwards the request to a Frontend Worker which in return will authenticate the caller and check function metadata to validate if the per function ...
Lambda is standalone service that doesn't need to be integrated with API Gateway. queryStringParameters
, body
, body mapping templates
, all of this is specific not to Lambda, but to Lambda - API Gateway integration.
If you are using Lambda with other services then the data is usually passed directly via event
object and there is not much of a reason to pass it in some other way.
For example, you can subscribe Lambda function to S3 bucket and use it to programatically process events such as file being uploaded to your bucket. In this case, information such as bucket name, object key, object data, metadata, ... will be passed directly via event
object.
And, when using Lambda with API Gateway, why would you want to use body mapping templates
to pass data to your Lambda function directly via event
object? Because you can reuse that function much easier for other purposes (if viable in your scenario), because your Lambda function will have much simpler interface, instead of one that is tight to API Gateway integration.
For example, you can have a function that performs some calculations on passed in numbers which you can call via API Gateway as well as to call it directly from your application. It will be much easier to work with such function if it expects event.x
and event.y
instead of some event.queryStringParameter.x
which might make zero sense outside of API Gateway.
Building on insights from Matus, I now can better answer the rest of the questions from above:
When setting up API Gateway with Lambda to send-through specific pieces of data you need to use a Mapping Template which gets setup in AWS API Gateway.
That mapping template itself is written in the Velocity Template Language (VTL) from the Apache Foundation. Ex for attaching only the "hello" data so it is accessible as const data = event.hello; // world
, ex:
{
"hello": $input.params('$hello')
}
Note: VTL is very powerful, the above is not a realistic example of usage but smallest amount of VTL just to communicate the idea, see here and here for more detail
Jumping through the interface to set that Mapping Template is a bit of an ordeal for a starter example so here's illustrated steps:
Warning: Be sure to double-check what is actually displayed in the editor, since there might occasionally occur some funky/unexpected behavior on the Mapping Template when you change the dropdown. Sometimes you also reach unrecoverable issues involving the mapping template, it's best when starting out to go ahead and delete the method and restart from API Gateway.
It all depends on how your Lambda is receiving its data.
The key insight from Matus is that this is all ultimately an implementation detail. Lambdas in this contrived example (ie set up a REST API via AWS Gateway, send a GET/POST to that API and have lambda do something with the data) can retrieve data 3 ways:
(1) URL parameters const data = event.queryStringParameters.hello; // world
(2) request body const data = event.body.hello; // world
(see note below)
(3) directly on the event object const data = event.hello; // world
Note on 2: This requires selecting (a) Lambda Proxy Integration instead of using a mapping template and in your code you'll need JSON.parse
the event body before accessing the data, see this answer for further details
All this depends on what it is being fed from API Gateway. In this specific example, I'm talking about doing a REST request to pass data on an API Gateway endpoint that then is processed by Lambda-- but lots of other services/triggers can send data to a Lambda script for analysis.
Other helpful resources:
https://kennbrodhagen.net/2015/12/06/how-to-create-a-request-object-for-your-lambda-event-from-api-gateway/
Getting json body in aws Lambda via API gateway
Could not parse request body into json: Unexpected character (\'-\' (code 45)) AWS Lambda + API + Postman
https://medium.com/@lakshmanLD/lambda-proxy-vs-lambda-integration-in-aws-api-gateway-3a9397af0e6d
https://github.com/valgaze/documentdb-fun
Ex. a toy Lambda function echos back whatever is POST'd:
let client = null; // Data outside function handler (ex a DB connection or an incrementing value) can change within the handler code below and persist between Lamba invocations so long as the container is still "warm", see more for details: https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html
exports.handler = async (event, context) => {
let data = {};
// Lambda Proxy Integration
if (event && event.body) {
try {
data = JSON.parse(event.body);
} catch(e) {
return {
statusCode: 400,
body: JSON.stringify({message: 'There was an error parsing the JSON data posted to this endpoint', error:e})
}
}
}
try {
// Echo back the data or perform transformations, pass on to another service, etc
const response = {
statusCode: 200,
body: JSON.stringify({message: 'Data submitted', data})
};
return response;
} catch(e) {
// Report errors related with connection, auth, DB write, etc
return {
statusCode: 409,
body: JSON.stringify({message: 'There was some type of catastrophic error', error:e})
}
}
};
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With