Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS step functions and optional parameters

I would like to have a default value for a parameter passed into a step function

e.g.,

"Parameters": {
   "foo.$": "$.foo" OR "bar" if "$.foo" not specified
}

is there a way to do this natively with JSONPath or do I have to use a choice + pass state?

I'd even settle for using choice/pass if there were a way to not break when a parameter is not specified in the input.

If I don't include "foo": "" in the input, I will get an error like "JSONPath ... could not be found in the input."

like image 314
wrschneider Avatar asked Nov 26 '19 17:11

wrschneider


People also ask

How do I use AWS step functions?

AWS Step Functions applies the InputPath field first, and then the Parameters field. You can first filter your raw input to a selection you want using InputPath, and then apply Parameters to manipulate that input further, or add new values.

What is the resultselector field in AWS step functions?

The ResultSelector field provides a way to manipulate the state’s result before ResultPath is applied. AWS Step Functions applies the InputPath field first, and then the Parameters field.

How do I Pass API parameters to an AWS Batch job?

In case your task state orchestrates an AWS Batch job, you'll easily pass all the relevant API parameters straight to the API actions of that service. The ItemsPath field is utilized in a Map state so you could select an array within the input. A Map state is used to iterate steps for every item within an array found in the input.

What is resultpath in AWS Lambda?

The Lambda function will only receive the JSON object {"who": "AWS Step Functions"} as an input. This particular ResultPath will tell the state machine to insert the Lambda function result in a node called "lambdaresult" as a data node's child within the original state machine input.


Video Answer


3 Answers

I'm solving this using a combination of the Choice and Pass states. Given that a state machine at least gets an empty input object, you can check if it has members present using the IsPresent comparison operator in the Choice state. If your variable of desire is not present, you can route to a Pass state to inject a default fallback object.

Example Input

{
  "keyThatMightNotExist": {
    "options": {
      "foo": "bar",
      "baz": false
    },
    "id": 1234
  }
}

State Machine Definition

{
  "Comment": "An example for how to deal with empty input and setting defaults.",
  "StartAt": "Choice State: looking for input",
  "States": {
    "Choice State: looking for input": {
      "Type": "Choice",
      "Choices": [
        {

Checking for existence and if so, also validate a child member:


          "And": [
            {
              "Variable": "$.keyThatMightNotExist",
              "IsPresent": true
            },
            {
              "Variable": "$.keyThatMightNotExist.id",
              "IsNull": false
            }
          ],

If the key variable is present and its child node "id" is true, skip the next state and hop over to the "State that works with $.keyThatMightNotExist"

          "Next": "State that works with $.keyThatMightNotExist"
        }
      ],
      "Default": "LoadDefaults"
    },

The following is where we inject our defaults:


    "LoadDefaults": {
      "Type": "Pass",
      "Result": {
        "id": 0,
        "text": "not applicable"
      },
      "ResultPath": "$.keyThatMightNotExist",
      "Next": "State that works with $.keyThatMightNotExist"
    },

At this point, there is an object to work with. Either from actual input, or by using the defaults:


    "State that works with $.keyThatMightNotExist": {
      "Type": "Succeed"
    }
  }
}

enter image description here

For further information, refer to the AWS Step Functions Developer Guide, choice-state-example

like image 190
Justus Kenklies Avatar answered Oct 19 '22 19:10

Justus Kenklies


This can be achieved by creating a Pass State in the starting of step function with the combination of "Result" and "ResultPath".

  • "Result" of Pass State will be the value of foo
  • "ResultPath" : "$.foo" in Pass State, so it's going to add foo variable in your pass input.

State Definition for the same will look something like this:

"Hello": {
      "Type": "Pass",
      "Result": "Added from Hello state",
      "ResultPath":"$.foo",
      "Next": "OtherStates"
    }

An example would be: Input to Hello state:

{
 "bar" : "From input" 
}

Output of Hello state:

{
 "bar" : "From input",
 "foo" : "Added from Hello state"
}

In this case it's going to add value of foo as "Added from Hello state". So to avoid breaking things because of no foo present in input, you will have to define this Pass State as the very first state or atleast before the state where you are going to use it.

P.S. This is applicable for the case when you just have to add a single default variable as you mentioned. For adding multiple default variable, I would recommend to create a Task State and achieve creating default variable using a lambda function.

like image 27
frosty Avatar answered Oct 19 '22 19:10

frosty


We had a similar issue that we resolved in a way that may be helpful dependant on what your SFN task will be doing. In our case it was a Lambda so we could handle the default behaviour within.

You can set a parameter to the "$" value that will take all the inputs provided to the SFN.

"Parameters": {
      "sfn_input.$": "$"
}

This "sfn_input" parameter will now be exposed to the stage and contain ALL inputs to the SFN. In our case, we could handle its optional presence in code.

This obviously only works if your task can eval the existence of the value. There could even be a concept of an "initialise" Lambda added at the start of your SFN to specifically handle this and add default values that pass back out to the SFN.

like image 4
Edge Avatar answered Oct 19 '22 20:10

Edge