DEV Community

Discussion on: Scaling Airflow – Astronomer Vs Cloud Composer Vs Managed Workflows For Apache Airflow

Collapse
 
bensums profile image
Ben Summers

Interesting. Never thought of problems scaling/customizing Airflow itself as just used it to schedule jobs on other platforms (e.g. dag could be: launch/acquire an EMR cluster, start the job, wait for the job to finish, kill the cluster (if our responsibility))