skhatri / airflow-by-example
Bunch of Airflow Configurations and DAGs for Kubernetes, Spark based data-pipelines. Scale inside Kubernetes using spark kubernetes master. Secure it with keycloak
☆23Updated 3 years ago
Alternatives and similar repositories for airflow-by-example
Users that are interested in airflow-by-example are comparing it to the libraries listed below
Sorting:
- A repository of sample code to show data quality checking best practices using Airflow.☆77Updated 2 years ago
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆168Updated last year
- Trino dbt demo project to mix and load BigQuery data with and in a local PostgreSQL database☆75Updated 3 years ago
- Astronomer Core Docker Images☆107Updated 11 months ago
- Pylint plugin for static code analysis on Airflow code☆94Updated 4 years ago
- ☆28Updated last year
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆35Updated last year
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆83Updated last year
- Enforce Best Practices for all your Airflow DAGs. ⭐☆99Updated this week
- Data validation library for PySpark 3.0.0☆33Updated 2 years ago
- Airflow training for the crunch conf☆105Updated 6 years ago
- ☆21Updated 3 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆87Updated 4 years ago
- ☆49Updated 3 years ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆27Updated 2 years ago
- Sample configuration to deploy a modern data platform.☆88Updated 3 years ago
- ☆1Updated 8 months ago
- Read Delta tables without any Spark☆47Updated last year
- Delta Lake Documentation☆49Updated 10 months ago
- The Python fake data producer for Apache Kafka® is a complete demo app allowing you to quickly produce JSON fake streaming datasets and …☆85Updated last year
- Airflow Providers containing Deferrable Operators & Sensors from Astronomer☆148Updated this week
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆70Updated 7 months ago
- Great Expectations Airflow operator☆163Updated this week
- PySpark data-pipeline testing and CICD☆28Updated 4 years ago
- ☆36Updated 2 months ago
- Creates simple data models on Snowflake to report dbt source freshness and tests☆26Updated last year
- A CLI tool to streamline getting started with Apache Airflow™ and managing multiple Airflow projects☆220Updated 2 weeks ago
- Fast iterative local development and testing of Apache Airflow workflows☆200Updated 3 weeks ago
- ☆50Updated 4 years ago
- A curated list of dagster code snippets for data engineers☆55Updated last year