gtoonstra / etl-with-airflow
ETL best practices with airflow, with examples
☆1,282Updated 2 years ago
Related projects: ⓘ
- A series of DAGs/Workflows to help maintain the operation of Airflow☆1,652Updated 3 months ago
- Dynamically generate Apache Airflow DAGs from YAML configuration files☆1,158Updated last week
- Guides and docs to help you get up and running with Apache Airflow.☆798Updated last year
- Curated list of resources about Apache Airflow☆3,653Updated 3 weeks ago
- A curated list of data engineering tools for software developers☆460Updated 7 years ago
- Airflow basics tutorial☆397Updated 3 years ago
- Docker Apache Airflow☆3,765Updated last year
- Apache Airflow tutorial☆922Updated last year
- Example DAGs using hooks and operators from Airflow Plugins☆333Updated 6 years ago
- A docker image and kubernetes config files to run Airflow on Kubernetes☆653Updated 5 years ago
- Implementing best practices for PySpark ETL jobs and applications.☆1,631Updated last year
- A plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces☆325Updated 3 years ago
- Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io☆1,866Updated last week
- Code for Data Pipelines with Apache Airflow☆696Updated last month
- Collect, aggregate, and visualize a data ecosystem's metadata☆1,732Updated last week
- pyspark methods to enhance developer productivity 📣 👯 🎉☆624Updated last week
- Helm Charts for the Astronomer Platform, Apache Airflow as a Service on Kubernetes☆461Updated this week
- A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow☆2,068Updated 9 months ago
- Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting…☆4,391Updated last week
- Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.☆3,252Updated last week
- Apache Airflow integration for dbt☆392Updated 4 months ago
- A plugin for Apache Airflow that allows you to edit DAGs in browser☆395Updated this week
- Airflow Unit Tests and Integration Tests☆254Updated last year
- A curated list of awesome ETL frameworks, libraries, and software.☆3,247Updated last month
- An Open Standard for lineage metadata collection☆1,708Updated this week
- PySpark test helper methods with beautiful error messages☆583Updated last week
- A boilerplate for writing PySpark Jobs☆392Updated 7 months ago
- Python API for Deequ☆704Updated last week
- An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.☆1,272Updated 4 years ago
- Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)☆184Updated 9 months ago