Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parallel States Merge the output in Step Function

Is it possible to have following kind of Step Function graph, i.e. from 2 parallel state output, one combined state:

enter image description here

If yes, what would json for this looks like? If not, why?

like image 620
hatellla Avatar asked Jan 09 '19 08:01

hatellla


People also ask

Can Step Functions run in parallel?

A Parallel state causes AWS Step Functions to execute each branch, starting with the state named in that branch's StartAt field, as concurrently as possible, and wait until all branches terminate (reach a terminal state) before processing the Parallel state's Next field.

How do you find the output of a step function?

To get the function output from a Step Function, you have to add a second method in API Gateway which will call the Step Function with the DescribeExecution action. The API Gateway client will have to call this periodically (poll) until the returned status is no longer "RUNNING".

What is state transition in step function?

Step Functions counts a state transition each time a step of your workflow is executed. You are charged for the total number of state transitions across all your state machines, including retries. The Step Functions free tier includes 4,000 free state transitions per month.

How do you pass data between Step Functions?

You can give AWS Step Functions initial input data by passing it to a StartExecution action when you start an execution, or by passing initial data using the Step Functions console . Initial data is passed to the state machine's StartAt state. If no input is provided, the default is an empty object ( {} ).


1 Answers

A parallel task always outputs an array (containing one entry per branch).

You can tell AWS step functions to append the output into new (or existing) property in the original input with "ResultPath": "$.ParallelOut" in your parallel state definition, but this is not what you seem to be trying to achieve.

To merge the output of parallel task, you can leverage the "Type": "Pass" state to define transformations to apply to the JSON document.

For example, in the state machine below, I'm transforming a JSON array...

[
  {
    "One": 1,
    "Two": 2
  },
  {
    "Foo": "Bar",
    "Hello": "World"
  }
]

...into a few properties

{
  "Hello": "World",
  "One": 1,
  "Foo": "Bar",
  "Two": 2
}

Transform an array into properties with AWS Step Functions

{
    "Comment": "How to convert an array into properties",
    "StartAt": "warm-up",
    "States": {
      "warm-up": {
        "Type": "Parallel",
        "Next": "array-to-properties",
        "Branches": [
          {
            "StartAt": "numbers",
            "States": {
              "numbers": {
                "Type": "Pass",
                "Result": {
                    "One": 1,
                    "Two" : 2
                },
                "End": true
              }
            }
          },
          {
            "StartAt": "words",
            "States": {
              "words": {
                "Type": "Pass",
                "Result": {
                    "Foo": "Bar",
                    "Hello": "World"
                },
                "End": true
              }
            }
          }
        ]
      },
      "array-to-properties": {
        "Type": "Pass",
        "Parameters": {
          "One.$": "$[0].One",
          "Two.$": "$[0].Two",
          "Foo.$": "$[1].Foo",
          "Hello.$": "$[1].Hello"
        },
        "End": true
      }
    }
}
like image 102
bounav Avatar answered Sep 18 '22 15:09

bounav