Metadata
- Author: Hubert Dulay
- Full Title:: Limits of the Event-Driven Orchestrator
- Category:: 🗞️Articles
- Document Tags:: workflow orchestrators Workflow orchestrators
- URL:: https://stkbailey.substack.com/p/limits-of-the-event-driven-orchestrator#%C2%A7streaming-workflow-builders
- Finished date:: 2023-04-10
Highlights
Referenced from Life After Orchestrators. It is a bit weird: who wanted to do streaming processing with an orchestrator?
This is essay #5 in the Symposium on Is the Orchestrator Dead or Alive? (View Highlight)
Airflow and Dagster are similar to CRON with additional enhancements (View Highlight)
With this understanding of orchestrators, I’ll try to use another event-driven orchestrator called Prefect. (View Highlight)
Prefect is an open-source orchestration platform that looks promising. It natively has a way to consume events (View Highlight)
orchestrators cannot replace today’s popular stream processors (View Highlight)
what is event-driven orchestration supposed to do? The answer is really notifications. Event-driven orchestrations are really notification consumers. They only trigger tasks and do not define the task itself. The tasks that orchestrators invoke could be a batch process that can handle large volumes of data but that data wouldn’t come from Kafka. Batch data comes from data stores that hold data at rest. (View Highlight)