airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

🔮 Build multimodal AI services via cloud native technologies · Neural Search · Generative AI · Cloud Native

Workflow engine for Kubernetes

Apache DolphinScheduler is the modern data workflow orchestration platform with powerful user interface, dedicated to solving complex task dependencies in the data pipeline and providing various types of jobs available `out of the box`

Free Data Engineering course!

PipelineAI Kubeflow Distribution

Build data pipelines, the easy way 🛠️

Docker Apache Airflow

Curated list of resources about Apache Airflow

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Elyra extends JupyterLab with an AI centric approach.

A series of DAGs/Workflows to help maintain the operation of Airflow

Data reliability tools for SQL- and Spark-accessible data

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

A Data Engineering & Machine Learning Knowledge Hub

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Dynamically generate Apache Airflow DAGs from YAML configuration files

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Example end to end data engineering project.

Just another Cron alternative with a Web UI, but with much more capabilities. It aims to solve greater problems.