skhatri / airflow-by-exampleLinks
Bunch of Airflow Configurations and DAGs for Kubernetes, Spark based data-pipelines. Scale inside Kubernetes using spark kubernetes master. Secure it with keycloak
☆23Updated 3 years ago
Alternatives and similar repositories for airflow-by-example
Users that are interested in airflow-by-example are comparing it to the libraries listed below
Sorting:
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆168Updated last year
- A repository of sample code to show data quality checking best practices using Airflow.☆77Updated 2 years ago
- Pylint plugin for static code analysis on Airflow code☆95Updated 4 years ago
- Airflow Providers containing Deferrable Operators & Sensors from Astronomer☆148Updated this week
- Creates simple data models on Snowflake to report dbt source freshness and tests☆26Updated last year
- Examples of various flow deployments for Prefect 1.0 (storage and run configurations)☆35Updated 3 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆88Updated 4 years ago
- Utility functions for dbt projects running on Spark☆34Updated 3 months ago
- Make simple storing test results and visualisation of these in a BI dashboard☆44Updated 2 months ago
- Enforce Best Practices for all your Airflow DAGs. ⭐☆101Updated this week
- Airflow training for the crunch conf☆105Updated 6 years ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆28Updated 2 years ago
- Sample configuration to deploy a modern data platform.☆88Updated 3 years ago
- A GitHub Action that makes it easy to use Great Expectations to validate your data pipelines in your CI workflows.☆80Updated last year
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆70Updated 8 months ago
- ☆75Updated this week
- A PyTest plugin to speed up your tests which depend on Snowflake sessions☆27Updated last week
- Pytest plugin for dbt core☆60Updated 4 months ago
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆35Updated last year
- ☆21Updated 4 years ago
- Repo for orienting dbt users to the Dagster asset framework☆54Updated 2 years ago
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆83Updated last year
- Great Expectations Airflow operator☆165Updated this week
- Delta Lake helper methods. No Spark dependency.☆23Updated 8 months ago
- ☆199Updated last year
- Astronomer Core Docker Images☆107Updated last year
- Delta-Lake, ETL, Spark, Airflow☆47Updated 2 years ago
- The shared semantic layer definitions that dbt-core and MetricFlow use.☆78Updated 2 months ago
- Rules based grant management for Snowflake☆40Updated 6 years ago
- Code examples showing flow deployment to various types of infrastructure☆107Updated 2 years ago