HTTP request body not getting to AWS lambda function via AWS API Gateway

Question

I have a very basic lambda function written in Scala deployed to AWS Lambda. The function works just fine when I test it via the AWS Lambda console.

Here's the function with some additional logging added for debug purposes.

package com.spacecorpshandbook.ostium.lambda.handler

import java.util

import com.google.gson.Gson
import temp.{ApiGatewayProxyResponse, Appointment, CancelResponse}

/**
  * Amazon Lambda handler adapter for the Cancellation application
  */
class CancellationHandler {

  def cancelAppointment(appointment: Appointment): ApiGatewayProxyResponse = {

    System.out.println("++++ appointmentId is: " + appointment.getAppointmentId)

    val apiGatewayProxyResponse = new ApiGatewayProxyResponse
    val cancelResponse = new CancelResponse

    cancelResponse.setMessage("Cancelled appointment with id " + appointment.getAppointmentId)

    val gson: Gson = new Gson

    apiGatewayProxyResponse.setBody(gson.toJson(cancelResponse))

    apiGatewayProxyResponse.setStatusCode("200")

    val headerValues = new util.HashMap[String, String]

    headerValues put("Content-Type", "application/json")

    apiGatewayProxyResponse.setHeaders(headerValues)

    System.out.println("+++++ message before returning: " + apiGatewayProxyResponse.getBody)

    apiGatewayProxyResponse
  }

}

I was concerned that the POJO input/outputs being Scala beans might have been causing issues so I implemented Java versions temporarily just to rule that out.

The Integration Request on AWS API gateway is setup by default as Resources ANY with Lambda Proxy Integration enabled. Note that in this configuration when I test from the AWS API Gateway console the data is transformed and goes in but does't make it all the way to the lambda function

Execution log for request test-request
Fri Dec 09 11:14:40 UTC 2016 : Starting execution for request: test-invoke-request
Fri Dec 09 11:14:40 UTC 2016 : HTTP Method: PUT, Resource Path: /cancel-appointment
Fri Dec 09 11:14:40 UTC 2016 : Method request path: {}
Fri Dec 09 11:14:40 UTC 2016 : Method request query string: {}
Fri Dec 09 11:14:40 UTC 2016 : Method request headers: {Content-Type= application/json}
Fri Dec 09 11:14:40 UTC 2016 : Method request body before transformations: {
    "applicationId": "asdfsfa"
}
Fri Dec 09 11:14:40 UTC 2016 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=****************************************************************************************************************************************************************************************************************************************************************************************************************************************5c044d, X-Amz-Date=20161209T111440Z, x-amzn-apigateway-api-id=l5tcmj0vlk,  Accept=application/json, User-Agent=AmazonAPIGateway_l5tcmj0vlk, Host=lambda.us-east-1.amazonaws.com, X-Amz-Content-Sha256=857a062940a7fbb8134bad1c007e9975a10bd8323c39f6040e797a98e87ea1f6, X-Amzn-Trace-Id=Root=1-584a9220-9cd537954952cca7daee32bf, Content-Type=application/json}
Fri Dec 09 11:14:40 UTC 2016 : Endpoint request body after transformations: {"resource":"/cancel-appointment","path":"/cancel-appointment","httpMethod":"PUT","headers":{"Content-Type":" application/json"},"queryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"accountId":"456204981758","resourceId":"xznq3u","stage":"test-invoke-stage","requestId":"test-invoke-request","identity":{"cognitoIdentityPoolId":null,"accountId":"456204981758","cognitoIdentityId":null,"caller":"456204981758","apiKey":"test-invoke-api-key","sourceIp":"test-invoke-source-ip","accessKey":"ASIAJ5D7KU524H7CTTTQ","cognitoAuthenticationType":null,"cognitoAuthenticationProvider":null,"userArn":"arn:aws:iam::456204981758:root","userAgent":"Apache-HttpClient/4.5.x (Java/1.8.0_102)","user":"456204981758"},"resourcePath":"/cancel-appointment","httpMethod":"PUT","apiId":"l5tcmj0vlk"},"body":"{\n    \"applicationId\": \"asdfsfa\"\n}","isBase64Encoded":false}
Fri Dec 09 11:14:40 UTC 2016 : Endpoint response body before transformations: {"statusCode":"200","headers":{"Content-Type":"application/json"},"body":"{\"message\":\"Cancelled appointment with id null\"}"}
Fri Dec 09 11:14:40 UTC 2016 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=adcadf25-be00-11e6-8855-75e96d772946, Connection=keep-alive, Content-Length=128, Date=Fri, 09 Dec 2016 11:14:39 GMT, Content-Type=application/json}
Fri Dec 09 11:14:40 UTC 2016 : Method response body after transformations: {"message":"Cancelled appointment with id null"}
Fri Dec 09 11:14:40 UTC 2016 : Method response headers: {Content-Type=application/json, X-Amzn-Trace-Id=Root=1-584a9220-9cd537954952cca7daee32bf}
Fri Dec 09 11:14:40 UTC 2016 : Successfully completed execution
Fri Dec 09 11:14:40 UTC 2016 : Method completed with status: 200

If I add a specific method, like POST and do not set it as a Lambda Proxy Integration I do indeed see the that the provide request body data makes it to the lambda function, is correctly de-serialized into my POJO and is returned

Execution log for request test-request
Fri Dec 09 11:22:02 UTC 2016 : Starting execution for request: test-invoke-request
Fri Dec 09 11:22:02 UTC 2016 : HTTP Method: POST, Resource Path: /cancel-appointment
Fri Dec 09 11:22:02 UTC 2016 : Method request path: {}
Fri Dec 09 11:22:02 UTC 2016 : Method request query string: {}
Fri Dec 09 11:22:02 UTC 2016 : Method request headers: {}
Fri Dec 09 11:22:02 UTC 2016 : Method request body before transformations: {
    "appointmentId" : "sfssdf"
}
Fri Dec 09 11:22:02 UTC 2016 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=****************************************************************************************************************************************************************************************************************************************************************************************************************************************a8dc41, X-Amz-Date=20161209T112202Z, x-amzn-apigateway-api-id=l5tcmj0vlk, Accept=application/json, User-Agent=AmazonAPIGateway_l5tcmj0vlk, Host=lambda.us-east-1.amazonaws.com, X-Amz-Content-Sha256=875dad4d4e05f8c12a7ca8aeaf69046d4153fc7f910e1eff1959cb011e8313a0, X-Amzn-Trace-Id=Root=1-584a93da-f841704d9feb371b31e41cb9, Content-Type=application/json}
Fri Dec 09 11:22:02 UTC 2016 : Endpoint request body after transformations: {
    "appointmentId" : "sfssdf"
}
Fri Dec 09 11:22:02 UTC 2016 : Endpoint response body before transformations: {"statusCode":"200","headers":{"Content-Type":"application/json"},"body":"{\"message\":\"Cancelled appointment with id sfssdf\"}"}
Fri Dec 09 11:22:02 UTC 2016 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=b4f5efce-be01-11e6-91c3-5b1e06f831e2, Connection=keep-alive, Content-Length=130, Date=Fri, 09 Dec 2016 11:22:02 GMT, Content-Type=application/json}
Fri Dec 09 11:22:02 UTC 2016 : Method response body after transformations: {"statusCode":"200","headers":{"Content-Type":"application/json"},"body":"{\"message\":\"Cancelled appointment with id sfssdf\"}"}
Fri Dec 09 11:22:02 UTC 2016 : Method response headers: {X-Amzn-Trace-Id=Root=1-584a93da-f841704d9feb371b31e41cb9, Content-Type=application/json}
Fri Dec 09 11:22:02 UTC 2016 : Successfully completed execution
Fri Dec 09 11:22:02 UTC 2016 : Method completed with status: 200

So everything looks great now, however when I actually do a real test against the AWS API URL from PostMan using HTTP method POST I get the response with null as the appointment Id and I can see in the CloudWatch logs that the appointmentId did not get set on the input Appointment object.

I feel like I'm missing something basic here. Any help would be greatly appreciated.

source code can be found here

Update

Resolved this problem by switching the lambda handler function to use Stream rather than attempting to serialize/deserialize JSON into POJOs. When using the API Gateway Lambda Proxy the input to handler is a complicated JSON structure that I didn't want to try and replicate as a Java/Scala class. It was easier to process the input as a stream, parse it into a JsonObject, and then convert the body of the message into my POJO using Gson or equivalent library. Sample handler below, you can also see a larger example here

class CancellationHandler {

  def cancelAppointment(request: InputStream, response: OutputStream, context: Context): Unit = {

    val logger = context.getLogger
    val parser: JsonParser = new JsonParser
    var inputObj: JsonObject = null
    val gson: Gson = new Gson

    try {

      inputObj = parser.parse(IOUtils.toString(request, "UTF-8")).getAsJsonObject

    } catch {

      case e: IOException =>

        logger.log("Error while reading request\n" + e.getMessage)
        throw new RuntimeException(e.getMessage)

    }

    val body: String = inputObj.get("body").getAsString
    val appointment: Appointment = gson.fromJson(body, classOf[Appointment])

    val apiGatewayProxyResponse = new ApiGatewayProxyResponse
    val cancelResponse = new CancelResponse

    cancelResponse.setMessage("Cancelled appointment with id " + appointment.getAppointmentId)

    apiGatewayProxyResponse.setBody(gson.toJson(cancelResponse))

    apiGatewayProxyResponse.setStatusCode("200")

    val headerValues = new util.HashMap[String, String]

    headerValues put("Content-Type", "application/json")

    apiGatewayProxyResponse.setHeaders(headerValues)

    val output: String = gson.toJson(apiGatewayProxyResponse)

    IOUtils.write(output, response, "UTF-8")
  }

}

Answer 1

The input shape for the Lambda proxy will be different than the shape for the regular non-proxy Lambda integration. This is important for your use case of course because you're using Java/Scala where you have to explicitly structure the input POJO.

Here's what the proxy input will look like:

{
  "resource": "\/pets",
  "path": "\/pets",
  "httpMethod": "POST",
  "headers": null,
  "queryStringParameters": null,
  "pathParameters": null,
  "stageVariables": null,
  "requestContext": {
    ...
    "stage": "test-invoke-stage",
    "requestId": "test-invoke-request",
    "identity": {
      ...
    },
    "resourcePath": "\/pets",
    "httpMethod": "POST"
  },
  "body": "{\n    \"foo\":\"bar\"\n}", <---- here's what you're looking for
  "isBase64Encoded": false
}

Docs: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html