Input and Ouput to ECS task in Step function

Question

Have previously worked with lambda orchestration using AWS step function. This has been working very well. Setting the result_path of each lambda will pass along arguments to subsequent lambda.

However, I now need to run a fargate task and then pass along arguments from that fargate task to subsequent lambdas. I have created a python script that acts as the entrypoint in the container definition. Obviously in a lambda function the handler(event, context) acts as the entrypoint and by defining a return {"return_object": "hello_world"} its easy to pass a long a argument to the next state of the state machine.

In my case though, I have task definition with a container definition created from this Dockerfile:

FROM python:3.7-slim

COPY my_script.py /my_script.py
RUN ln -s /python/my_script.py /usr/bin/my_script && \
chmod +x /python/my_script.py

ENTRYPOINT ["my_script"]

Hence, I am able to invoke the state machine and it will execute my_script as intended. But how do I get the output from this python script and pass it along to another state in the state machine?

I have found some documentation on how to pass along inputs, but no example of passing along outputs.

Answer 1

To get output from an ECS/Fargate task, I think you have to use the Task Token Integration instead of Run Job (Sync) which is usually recommended for Fargate tasks. You can pass the token as a container override ("TASK_TOKEN": "$$.Task.Token"). Then inside your image you need some logic like this:

client = boto3.client('stepfunctions')
client.send_task_success(
    taskToken=os.environ["TASK_TOKEN"],
    output=output
)

to pass it back.