Using Timers/Signals to allow human intervention in AWS SWF Workflow

Question

Here's the scenario. A user uploads an Excel file and this kicks off a workflow which validates the file, transforms it into a few different files, then performs an update to a database based on the transforms. After the uploads, the results need to be reviewed by team member before the flow can continue.

I'm using Ruby and have discovered that Signals and Timers are the way to achieve this in SWF. However, the Ruby examples are lacking or non-existent and I need a little help understanding how this would work using Ruby.

Ny understanding so far is that a Timer activity is scheduled which basically pauses the flow until either the timer expires (at which point I could cancel the workflow or email the staff and set another timer) or a signal is sent to the workflow to start the next step. The Decider would handle the signal and then kick off the appropriate activity.

Any thoughts or direction to other sources would be much appreciated.

Thanks, Thomas

Answer 1

It's somewhat difficult to provide an "answer", given you didn't really ask a specific question. I'm in agreement with you that using a Timer and Signals is what you want.

You don't specify how the team gets notified about the review. I'll assume that you notify them by email and direct them to some website where they can review the changes, and then click on a link to either Approve or Don't Approve. Clicking the link to Approve will send a request to a web server that will "signal" SWF that the review has been approved. Clicking the link to Don't Approve will "signal" SWF that the review has not been approved. You mention that you want to renotify the team (or perhaps escalate to the manager) if no one has taken action on the review. Let's say this renotification happens after 48 hours. After the renotication, you grant them another 72 hours before assumming Don't Approve.

Here's how your workflow looks like to me:

1. User uploads file and kicks off a workflow
2. Decider Task schedules "TransformActivity"
3. TransformActivity runs, transforms the data into different files, and completes successfully
4. Decider Task schedules "UpdateDatabaseActivity"
5. UpdateDatabaseActivity runs, updates the database, and completes successfully
6. Decider Task schedules "EmailTeamActivity"
7. EmailTeamActivity runs, emails the team, and completes successfully
8. Decider Task schedules a Timer for 48 hours.

If a signal indicating Approve or Don't Approve is received within 48 hours:

1. Decider Task schedules the "RecordFinalDecisionActivity"
2. RecordFinalDecisionActivity will run, record the Approve (or Don't Approve) into the database, and complete successfully.
3. Decider Task will then close the workflow because it's done.

If no signal is received and the timer fires (after 48 hours):

1. Decider Task schedules the "EmailTeamAndManagerActivity"
2. EmailTeamAndManagerActivity runs, emails the team and manager, and completes successfully.
3. Decider Task schedules another timer for 72 hours.

If a signal indicating Approve or Don't Approve is received within the additional 72 hours given:

1. Repeat the same logic as the section "If a signal indicating Approve or Don't Approve is received within 48 hours".

If no signal is received and the timer fires (after the additional 72 hours):

1. At this point, the workflow can assume it was a Don't Approve, schedule the "RecordFinalDecisionActivity" and close the workflow once that activity completes.

The reason why you don't want to have a "review" activity is because that task gets scheduled and then some activity worker needs to reply success. How would that work? When someone clicks the Approve or Don't Approve link, the request to the webserver would have to pull down the activity from the task list. However, if the task list has multiple activities, SWF just gives out any one of them. It might not get the right one. Now, you could argue that you could schedule the different reviews across different task lists, but that's just cumbersome and tedious.

Signals are done to indicate an "external" event, which this very much is. The SWF documentation on Signals does a great job on talking about Signals. Here's the SWF documentation on how to use Timers and Signals. As for the particulars on how to use SWF and Ruby, I can't really help you there. I've only used SWF with Java by using the AWS Flow Framework.

Answer 2

1. user upload excel file, does "StartWorkflowExecution", that queues a decision task
2. decision worker notice flow is new / "stage one", it schedules "transform file" activity task
3. activity worker picks up task, and does the "transform file" activity, when done does "RespondActivityTaskCompleted" with a result of "transformations done", that queues a decision task
4. decision worker picks up decision task, notices the transformations are done and schedule a new activity task
5. activity worker picks up activity task, notices it's for a team member (according to the instructions given by the decision worker when scheduling the activity task), team member gets notified, somehow perform his action, then somehow notifies the activity worker which will reply "RespondActivityTaskCompleted"

I don't see the need for a Timer or a Signal, it's just plain flow. Those two concepts are useful if you want recurring events, timeouts, and/or interrupting the flow.

Please note that you can differentiate activity workers by using task lists (for example activity workers for automated work vs activity workers for human participants, whatever).