how to access error information in Step Function from previous Function in Catch state

Question

I am wondering how to access the cause of my custom exception raised inside my lambda function. I need to access it at the end of my Step Functions workflow, as shown below.

The diagram below is an example of a failed execution. The error (error-info object, with its' own Error and Cause sections) is found in the output of ParseTextractOutput, but I am wondering how to access it in OutputNotFound as shown below.

Step Functions Diagram

Output

The output of ParseTextractOutput is

{
  "event"...
  "error-info": {
    "Error": "OutputNotFoundException",
    "Cause": "{\"errorMessage\": \"Contents of Textracted file: {...}}"
    }
  }
}

I'd like to access this data somehow in these fields (of the Step Functions definition):

...
  "States": {
    "OutputNotFound": {
      "Type": "Fail",
      "Error": "<useful stuff here, like $.error-info.Error or something>",
      "Cause": "<useful stuff here, like $.error-info.Cause or something>"
    },
...
    "ParseTextractOutput": {
      "Type": "Task",
      "Resource": "functionARN",
      "Catch": [
        {
          "ErrorEquals": ["OutputNotFoundException"],
          "ResultPath": "$.error-info",
          "Next": "OutputNotFound"
        }
      ],
      "End": true
    }

Python Code

Here's the relevant code for the Function ParseTextractOutput.

class OutputNotFoundException(Exception):
  pass

...

try:
  blocks = data['Blocks']
except KeyError as e:
  raise OutputNotFoundException('Contents of Textracted file: {}'.format(data))

Answer 1

Including a section of the workflow as one single branch of a Parallel state definition might be useful to define one exception scope, and take a new path with the exception details, like explained in this post:

{
  "Comment": "Better error handling",
  "StartAt": "ErrorHandler",
  "States": {
    "ErrorHandler": {
      "Type": "Parallel",
      "Branches": [
        {
          "StartAt": "Hello",
          "States": {
            "Hello": {
              "Type": "Pass",
              "Result": "Hello",
              "Next": "World"
            },
            "World": {
              "Type": "Pass",
              "Result": "World",
              "Next": "Foo"
            },
            "Foo": {
              "Type": "Pass",
              "Result": "World",
              "Next": "Bar"
            },
            "Bar": {
              "Type": "Pass",
              "Result": "World",
              "End": true
            }
          }
        }
      ],
      "Catch": [
        {
          "ErrorEquals": [
            "States.ALL"
          ],
          "ResultPath": "$.error",
          "Next": "Send Failure Message"
        }
      ],
      "Next": "Job Succeeded"
    },
    "Job Succeeded": {
      "Type": "Succeed"
    },
    "Send Failure Message": {
      "Type": "Pass",
      "Next": "Fail Workflow"
    },
    "Fail Workflow": {
      "Type": "Fail"
    }
  }
}

Answer 2

At the moment (with the current version of https://states-language.net/spec.html) Fail.Error and Fail.Cause cannot be dynamic. The input that is passed to Fail state is ignored and fix strings are used for error and cause.

We can view Fail as a point in the execution to announce a fix message, indicate end of execution with an error and exit.

This means any processing has to be done before these announcement points. As @frosty mentioned in the comments a Choice state can be useful.

Alternative 1: using Choice

Here is an example:

Let's say I have this Python code in my Lambda function:

class OutputNotFoundException(Exception):
  pass



def lambda_handler(event, context):
    raise OutputNotFoundException('Error message A')

When function returns, the output will be a JSON like this:

{
  "Error": "OutputNotFoundException",
  "Cause": "{\"errorMessage\":\"Error message A\",\"errorType\":\"OutputNotFoundException\",\"stackTrace\":[\"...\\n\"]}"
}

Notice how "Cause" is another JSON which is string encoded. We can convert OutputNotFound to Pass and use intrinsic function StringToJson() to convert the encoded string to normal JSON for easier processing later:

    "OutputNotFound": {
      "Type": "Pass",
      "Parameters": {
        "details.$": "States.StringToJson($.Cause)"
      },
      "Next": "Error message?"
    },

Now we have an output like this:

{
  "details": {
    "errorMessage": "Error message A",
    "errorType": "OutputNotFoundException",
    "stackTrace": ["...\n"]
  }
}

Next state will be a Choice which looks into $.details.errorMessage to decide a proper Fail state:

    "Error message?": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.details.errorMessage",
          "StringEquals": "Error message A",
          "Next": "Error A"
        },
        {
          "Variable": "$.details.errorMessage",
          "StringEquals": "Error message B",
          "Next": "Error B"
        }
      ],
      "Default": "Unknown Error"
    },

Each choice is now pointing to a normal Fail state to announce a fix string:

    "Error A": {
      "Type": "Fail",
      "Error": "OutputNotFoundException",
      "Cause": "OutputNotFoundException of type A happened"
    },

Alternative 2: End with Pass

If your intention is to have the exact error message as output of your execution for later logging/processing one way can be leaving at the Pass state:

    "OutputNotFound": {
      "Type": "Pass",
      "Parameters": {
        "details.$": "States.StringToJson($.Cause)",
        "isError": true
      },
      "End": true
    }

The downside of this solution is, of course, the execution ends with a successful status and we need to process output to discover there was an error (hence the extra isError field above)