AWS Step Function - how to overwrite a single input string field from a Lambda output

Question

I'm having difficulties overwriting a single field of an AWS Step Function input using the output of a Lambda function.

The Step Function definition:

{
  "Comment": "A Hello World example demonstrating various state types of the Amazon States Language",
  "StartAt": "ModifyInput",
  "States": {
    "ModifyInput": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "ResultPath": "$.token",
      "ResultSelector": {
        "token.$": "$.Payload.token"
      },
      "End": true,
      "TimeoutSeconds": 180,
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-west-2:639294609160:function:dummy-output",
        "Payload": {
          "Payload.$": "$"
        }
      }
    }
  }
}

The Lambda definition (Python):

def handle_request(event, context):
output = {
    "token": "1111-1111-1111-1111",
    ... // other data here
}
return output

The Step function is executed with a simple JSON input:

{
    "token": "0000-0000-0000-0000",
    "service_url": "http://www.test.com"
}

Taking a look at the Step Function definition I expect to have the "token" field to be overwritten from "0000-0000-0000-0000" to "1111-1111-1111-1111" having the ResultSelector returning only the token part of the Lambda output as a string.

However it turns out the ResultSelector actually returns the Lambda's output not as a single string value but as a JSON object with the value we need as a key-value pair. This results in the following final state of the Step Function:

{
  "output": {
    "token": {
      "token": "1111-1111-1111-1111"
    },
    "service_url": "http://www.test.com"
  },
  "outputDetails": {
    "truncated": false
  }
}

Please note that there's no way we can simply return a string from the Lambda itself. The Lambda will always return a JSON, we need a solution in AWS using paths maybe. Furthermore, even if the Lambda returned the string value straight (instead of object) the ResultSelector still makes it an object in AWS Step Function.

Is there a way the token field is overwritten by the string value only instead of a whole JSON object in a way to make the final output of the Step Function:

{
  "output": {
    "token": "1111-1111-1111-1111",
    "service_url": "http://www.test.com"
  },
  "outputDetails": {
    "truncated": false
  }
}

Answer 1

I have been struggling with the same issue and haven't found a proper solution yet, but here is a way-out I found only today:

1. Create a Pass-state after the lambda task call https://i.sstatic.net/V6cqW.png

2. Now send the elongated output in the input parameters as follow: https://i.sstatic.net/RTgNT.png

3. In the output tab configure as follows: https://i.sstatic.net/PyBNX.png

This should give you the output in your desired format.

Cons: Not efficient both in terms of no. of states (thus affecting cost - even if very minimal) and the extra effort required which could have been done directly in the lambda trigger state

Hope it helps!