Menu
I'm looking for an orchestration framework/engine/toolkit with which to replace/upgrade an existing software, mainly because of scalability limitations. By orchestration I mean asynchronous and distributed execution of generic tasks and workflows.
More specifically the requirements are pretty much these:
The existing system is an old fashion monolithic service (in Java) that has most of that, including the execution logic itself which should remain as untouched as possible.
Does anyone have experience with a similar problem? It seems to me it's supposed to be pretty common, would be strange if I have to implement it myself entirely. I found some questions here (like this and this) discussing the theory of orchestration and choreography systems, but not real examples of tools implementing it. Also I think we're not exactly talking about microservices - the tasks are not prolonged and heavy, they're just many, running in the background executing short jobs of many types. I wouldn't create a service for every job type.
I'm also not looking for cloud and container services at this point - to my understanding the deployment is a different issue.
The closest I got is the Netflix Conductor engine, which answers most of the requirements by running an orchestration server that manages tasks implemented in servlets (or any web services in any language - a plus). However it seems like it's built mainly for arranging heavy tasks in a workflow rather than running a huge number of small tasks, which makes me wonder what would be the overhead of invoking many small tasks in servlets for example.
Does anyone have experience or any input on the Conductor or other tools I could use? Or even my entire approach to the problem?
EDIT: I realize it's kind of a "research advice needed" so let's put it simply in 3 questions:
677
The Orchestration Framework (OF) is a tool that automates the logistics process. A configurable framework governed by business rules, the Orchestration Framework is used to determine the business scenarios in question. The Orchestration Framework (or OF) is a configurable tool that automates logistics processes.
Orchestration and scheduling tools strive to eliminate silos, streamline processes and automate repetitive tasks so that IT departments can move quickly and efficiently. While similar to automation, orchestration goes beyond the scope of automation by focusing on workflows or processes rather than simple tasks.
A microservice orchestration workflow is an architectural method of coordinating microservices for software systems and applications, in which loosely coupled services receive commands from a central controller, referred to as the orchestrator.
Acts as the backbone of your businessApache Kafka works in multi-step intermediator. It receives data from the source system and makes it available to target systems in real-time.
The main competitor of Netflix Conductor is Temporal Workflow. It scales better and is more developer-friendly by using code instead of JSON DSL to implement the orchestration logic.
It also works OK with the fine-grained tasks by implementing specific optimizations (local activities) that allow batching multiple small tasks into a single database update.
Temporal has been production hardened for over five years at Uber, Coinbase, HashiCorp, Dagadog, Stripe, and hundreds of other companies.
90
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
