Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a way of running AWS Step Functions locally when defined by CDK?

AWS Step Functions may be run in a local Docker environment using Step Functions Local Docker. However, the step functions need to be defined using the JSON-based Amazon States Language. This is not at all convenient if your AWS infrastructure (Step Functions plus lambdas) is defined using AWS CDK/CloudFormation.

Is there a way to create the Amazon States Language definition of a state machine from the CDK or CloudFormation output, such that it’s possible to run the step functions locally?

My development cycle is currently taking me 30 minutes to build/deploy/run my Lambda-based step functions in AWS in order to test them and there must surely be a better/faster way of testing them than this.

like image 210
John Avatar asked Aug 10 '20 22:08

John


People also ask

What is AWS step function & CDK & Sam?

This is a note on AWS step function & CDK & SAM local & miscellaneous subjects. This is a note on AWS step function & CDK & SAM local & miscellaneous subjects. AWS Step Functions is a "serverless" function orchestrator that makes it easy to sequence AWS Lambda functions and multiple AWS services into business-critical applications.

What is AWS step functions local?

You can now use a local version of AWS Step Functions to develop and test your workflows. AWS Step Functions Local is a downloadable version of Step Functions that lets you develop and test applications using a version of Step Functions running in your own development environment.

Can I run AWS step functions in a local Docker environment?

AWS Step Functions may be run in a local Docker environment using Step Functions Local Docker. However, the step functions need to be defined using the JSON-based Amazon States Language. This is not at all convenient if your AWS infrastructure (Step Functions plus lambdas) is defined using AWS CDK/CloudFormation.

How to unit test step function graphs in AWS CDK?

There is no good way to to unit testing of the graphs being created and we have to rely on running the graph with mocked/some implementation of lambda functions on aws account. With launch of AWS CDK, is it possible to have the step function graphs also defined in high level language and do unit testing, mocking etc?


2 Answers

We have been able to achieve this by the following:

Download:

https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local.html

To run step functions local, in the directory where you extracted the local Step Function files run:

java -jar StepFunctionsLocal.jar --lambda-endpoint http://localhost:3003

To create a state machine, you need a json definition (It can be pulled from the generated template or can get the toolkit plug in for Vs code, type step functions, select from a template and that can be your starter. Can also get it from the AWS console in the definition tab on the step function.

Run this command in the same directory as the definition json:

aws stepfunctions --endpoint http://localhost:8083 create-state-machine --definition "cat step-function.json" --name "local-state-machine" --role-arn "arn:aws:iam::012345678901:role/DummyRole"

You should be able to hit the SF now (hopefully) :)

like image 83
Matthew Dorrian Avatar answered Sep 17 '22 14:09

Matthew Dorrian


You can use cdk watch or the --hotswap option to deploy your updated state machine or Lambda functions without a CloudFormation deployment.

https://aws.amazon.com/blogs/developer/increasing-development-speed-with-cdk-watch/

If you want to test with Step Functions local, cdk synth generates the CloudFormation code containing the state machine's ASL JSON definition. If you get that and replace the CloudFormation references and intrinsic functions, you can use it to create and execute the state machine in Step Functions Local.

How some people have automated this:

  • https://nathanagez.com/blog/mocking-service-integration-step-functions-local-cdk/
  • https://github.com/kenfdev/step-functions-testing
like image 39
adamwong Avatar answered Sep 18 '22 14:09

adamwong