Why does my AWS Lambda Function return "Invalid JSON" error?

Question

I have a lambda function that I wrote a few days ago that was acting totally fine when tested. After going to test it today (without changing any of the code), I receive the following error: "Invalid lambda function output : Invalid JSON".

Here is the function code (Node.js 10.x):

const AWS = require("aws-sdk");
const joi = require("@hapi/joi");

const Cognito = new AWS.CognitoIdentityServiceProvider();

exports.handler = async (event) => {
    // NOTE: Cognito expects Username to be the user's email

    // Vars
    const userPoolId = process.env.COGNITO_USER_POOL_ID;
    const {email : UNSAFE_EMAIL, language : UNSAFE_LANGUAGE = "en-US"} = event;

    // Normalize email and language
    const UNSAFE_TRIMMED_EMAIL = UNSAFE_EMAIL.trim();
    const UNSAFE_TRIMMED_LANGUAGE = UNSAFE_LANGUAGE.trim();

    // Validate UNSAFE_INPUTS
    const languageRegex = /^[a-z]{2}-[A-Z]{2}$/;

    const schema = joi.object().keys({
        email: joi.string().trim().email({minDomainSegments: 2}).required(),
        language: joi.string().trim().min(2).max(5).regex(languageRegex).required()
    });

    const validationResult = joi.validate({
        email: UNSAFE_TRIMMED_EMAIL,
        language: UNSAFE_TRIMMED_LANGUAGE
    }, schema);

    if(validationResult.error) {
        console.log(JSON.stringify(validationResult.error, null, 2));
        return {
            statusCode: 400,
            body: JSON.stringify({
                error: true,
                error_message: "Invalid"
            })
        }
    }

    // Validation successful, change variable names to reflect
    const VALIDATED_EMAIL = UNSAFE_TRIMMED_EMAIL;
    const VALIDATED_LANGUAGE = UNSAFE_TRIMMED_LANGUAGE;

    // Cognito params
    // Username is the user's email
    // email is also required in UserAttributes in order to send confirmation
    // DesiredDeliveryMediums is required to send confirmation
    const params = {
        UserPoolId: userPoolId,
        Username: VALIDATED_EMAIL,
        UserAttributes: [
            {
                Name: "email",
                Value: VALIDATED_EMAIL
            },
            {
                Name: "custom:language",
                Value: VALIDATED_LANGUAGE
            } 
        ],
        DesiredDeliveryMediums: ["EMAIL"]
    }

    // Attempt to create user in Cognito
    try {
        const authRes = await Cognito.adminCreateUser(params).promise();
        console.log("Success: ", JSON.stringify(authRes, null, 2));
        return {
            statusCode: 200,
            body: JSON.stringify({
                success: true
            })
        }
    } catch(err) {
        console.log("Error: ", JSON.stringify(err, null, 2));
        return {
            statusCode: 400,
            body: JSON.stringify({
                error: true,
                error_message: err.message
            })
        }
    }
};

Running the tests, I get the expected error message when passing in badly formatted event data, and I get a Cognito error when attempting to create a user with the same email twice. Again, this is expected. However, when passing in a valid email with no users in the user pool I get the following as my response (formatted for readability):

Response:
{
  "statusCode": 400,
  "body": {
    "error": true,
    "error_message": "Invalid lambda function output : Invalid JSON"
  }
}

Checking in the Cognito User Pool that this function connects to, I see that a user has been successfully created. Yet, no email has been sent to the email address as was happening a few days ago.

All that is logged is information saying that I have an invalid JSON error, there is no authRes logged at all. When removing the call to Cognito and the corresponding console.log call, the try block runs successfully. So the issue is with the call to Cognito.

But why is this code failing today when it was working perfectly a few days ago? That is the part that is making me very frustrated.

Answer 1

The issue wasn't with this lambda function at all. It was an issue with AWS and the lambda function I was using as a Custom Message Trigger for Cognito User Pools. Here is what went wrong:

Per the AWS docs, the event data provided to the Custom Message Trigger lambda is of the following form for the adminCreateUser function call:

{
  "version": 1,
  "triggerSource": "CustomMessage_AdminCreateUser",
  "region": "<region>",
  "userPoolId": "<userPoolId>",
  "userName": "<userName>",
  "callerContext": {
      "awsSdk": "<calling aws sdk with version>",
      "clientId": "<apps client id>",
      ...
  },
  "request": {
      "userAttributes": {
          "phone_number_verified": false,
          "email_verified": true,
           ...
      },
      "codeParameter": "####",
      "usernameParameter": "username"
  },
  "response": {
      "smsMessage": "<custom message to be sent in the message with code parameter and username parameter>"
      "emailMessage": "<custom message to be sent in the message with code parameter and username parameter>"
      "emailSubject": "<custom email subject>"
  }
}

And it is expected that the data returned from the Custom Message Trigger lambda is of the same form as the event - only with the response object changed.

So this is what I wrote for the lambda:

const email_message = require("./email_message");

exports.handler = async (event) => {
    // Vars
    const {codeParameter, usernameParameter} = event.request;
    console.log("Cognito Event: ", event);

    // Check that codeParameter equals "####" and usernameParameter equals "username"
    // This is to ensure that no compromised values are entered into the html
    if(!(codeParameter === "####" && usernameParameter === "username")) {
        return null;
    }


    const newRes = {
        smsMessage: `Welcome: confirmation code is ${codeParameter} and username is ${usernameParameter}`,
        emailMessage: email_message({codeParameter, usernameParameter}),
        emailSubject: "Welcome To Our Site"
    }

    return {...event, response: newRes};
};

And this worked when tested a few days ago because the event object was of the form above. What had happened is that AWS sneakily changed the content of the codeParameter and usernameParameter fields to the following:

{
    ...
    "codeParameter": "{####}",
    "usernameParameter": "{username}",
    ...
}

So the lambda function was returning null as these strings didn't pass validation - and null isn't valid JSON.

So the temporary fix is to validate these new strings instead. However, this raises some concerns. Why is AWS changing the event object without so much as an update to the docs all of a sudden? Second, how should I validate that these strings are safe to inject in a customer's email address? I know that I can sanitize the usernameParameter but how about the codeParameter since it could very likely contain dangerous characters such as < > & ' " since it is a password generated with random symbols? If generating the password myself I can be sure that it won't contain data from a malicious actor so there is no need to sanitize. But if it's coming from AWS, who's to say that somehow these values aren't compromised? Hence why I added the validation step in the first place that was supposed to fail in the case those values had been changed. Which is exactly what happened.

So in short, all of my code behaved as expected. AWS changed their event object all of a sudden without notice.