Menu
At the moment I am investigating the possibility and the proper way of migrating complex web applications from AWS to GCP. There is actually no issues with mapping general compute and networking services from one provider to another, but I wonder if GCP has a service similar to AWS Step Functions? I've already taken a look at Google Dataflow and Google Cloud Tasks. The second one seems to be something like that, but I am not sure if it's the optimal solution.
So the question is what service from google provides same functionality as AWS Step Functions? And if there is no such - then combination of which services would you recommend to achieve effective orchestration of distributed tasks (primarily cloud functions). Thanks!
400
Cloud Dataflow frees you from operational tasks like resource management and performance optimization. AWS Step Functions belongs to "Cloud Task Management" category of the tech stack, while Google Cloud Dataflow can be primarily classified under "Real-time Data Processing".
Depending on your use case, you can have Step Functions call AWS services, such as Lambda, to perform tasks. You can create workflows that process and publish machine learning models. You can have Step Functions control AWS services, such as AWS Glue, to create extract, transform, and load (ETL) workflows.
Amazon Web Services (AWS) is a collection of digital infrastructure services that developers can leverage when developing their applications. The services include computing, storage, database, and application synchronization (messaging and queuing).
Build distributed applications using visual workflows. AWS Step Functions lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly. Using Step Functions, you can design and run workflows that stitch together services such as AWS Lambda and Amazon ECS into feature-rich applications.
2021 Update
As Brian de Alwis noted below, since this answer was written Cloud Workflows is now generally available and is functionally similar to Step Functions.
2019 Answer
As far as I'm aware there's nothing specifically like Step Functions, but I have two strategies for creating these types of micro-service systems on Google Cloud.
Strategy 1: Cloud Run/Cloud Functions with Pub/Sub
Here I'd create microservices using Cloud Run or Cloud Functions and subscribe these functions to Pub/Sub topics. That means that when Function A executes and completes it's work, it publishes a message to a specific topic with a data packet that any function subscribed to it will receive and execute.
For example you could create two topics named FunctionASuccess and FunctionAError and create two separate functions that subscribe to one or the other and handle the success and error use cases.
Strategy 2: Firebase Functions with Firestore/Realtime Database
Similarly to above I create Firebase Functions that watch for changes in Firestore or in the RTDB.
So Function A executes and completes its task, it saves a document to the FunctionAResults collection in Firestore or RTDB. Functions that are subscribed to changes in the FunctionAResults collection are then executed and take it to the next step.
They both work reliably so I have no preference, but I typically go with the 2nd strategy if I'm utilizing other Firebase services.
81
Cloud Workflows was announced at Cloud Next On Air 2020.
29
You're looking for Cloud Composer. It's based on the open-source library Apache Airflow which allows you to define and orchestrate workflows in a similar way to step functions.
23
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
