AWS - Step functions, use execution input within a TuningStep

Question

I've written a simple AWS step functions workflow with a single step:

from stepfunctions.inputs import ExecutionInput
from stepfunctions.steps import Chain, TuningStep
from stepfunctions.workflow import Workflow
import train_utils


def main():
    workflow_execution_role = 'arn:aws:iam::MY ARN'
    execution_input = ExecutionInput(schema={
        'app_id': str
    })
    estimator = train_utils.get_estimator()
    tuner = train_utils.get_tuner(estimator)

    tuning_step = TuningStep(state_id="HP Tuning", tuner=tuner, data={
        'train': f's3://my-bucket/{execution_input["app_id"]}/data/'},
                             wait_for_completion=True,
                             job_name='HP-Tuning')

    workflow_definition = Chain([
        tuning_step
    ])

    workflow = Workflow(
        name='HP-Tuning',
        definition=workflow_definition,
        role=workflow_execution_role,
        execution_input=execution_input
    )
    workflow.create()


if __name__ == '__main__':
    main()

My goal is to have the train input pulled from the execution JSON provided at runtime. When I execute the workflow (from the step functions console), providing the JSON {"app_id": "My App ID"} the tuning step does not get the right data, instead it gets a to_string representation of the stepfunctions.inputs.placeholders.ExecutionInput. Furthermore when looking at the generated ASL I can see that the execution input was rendered as a string:

... 
"DataSource": {
   "S3DataSource": {
   "S3DataType": "S3Prefix",
   "S3Uri": "s3://my-bucket/<stepfunctions.inputs.placeholders.ExecutionInput object at 0x12261f7d0>/data/",
    "S3DataDistributionType": "FullyReplicated"
    }
},
...

What am I doing wrong?

Update: As mentioned by @yoodan the SDK is probably behind, so I'll have to edit the definition before calling create. I can see there is a way to review the definition before calling create, but can I modify the graph definition? how?

Answer 1

The python SDK for step functions generates corresponding code, we need a string concatenation / format built into the Amazon States Language to accomplish what you desire.

Recently in August 2020, Amazon States Language introduced built-in functions such as string format into it's language spec. https://states-language.net/#appendix-b

Unfortunately, the python SDK is not up to date and does not support the new changes.

As a work around, maybe manually modify the definition before calling workflow create?