skhatri / airflow-by-exampleLinks
Bunch of Airflow Configurations and DAGs for Kubernetes, Spark based data-pipelines. Scale inside Kubernetes using spark kubernetes master. Secure it with keycloak
☆23Updated 3 years ago
Alternatives and similar repositories for airflow-by-example
Users that are interested in airflow-by-example are comparing it to the libraries listed below
Sorting:
- A repository of sample code to show data quality checking best practices using Airflow.☆77Updated 2 years ago
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆70Updated 9 months ago
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆169Updated last year
- Creates simple data models on Snowflake to report dbt source freshness and tests☆26Updated 2 years ago
- Pylint plugin for static code analysis on Airflow code☆95Updated 4 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆88Updated 4 years ago
- Astronomer Core Docker Images☆107Updated last year
- ☆1Updated 9 months ago
- Execution of DBT models using Apache Airflow through Docker Compose☆116Updated 2 years ago
- ☆49Updated 3 years ago
- Data-aware orchestration with dagster, dbt, and airbyte☆31Updated 2 years ago
- Great Expectations Airflow operator☆166Updated last week
- A simple and easy to use Data Quality (DQ) tool built with Python.☆50Updated last year
- Airflow Providers containing Deferrable Operators & Sensors from Astronomer☆149Updated last week
- PySpark data-pipeline testing and CICD☆28Updated 4 years ago
- ☆50Updated 4 years ago
- New generation opensource data stack☆69Updated 3 years ago
- Data pipeline with dbt, Airflow, Great Expectations☆163Updated 3 years ago
- Pipeline definitions for managing data flows to power analytics at MIT Open Learning☆43Updated this week
- Airflow training for the crunch conf☆105Updated 6 years ago
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆35Updated last year
- Utility functions for dbt projects running on Spark☆34Updated 4 months ago
- A skeleton project for testing Airflow code☆20Updated 3 years ago
- pytest plugin to run the tests with support of pyspark☆86Updated last month
- Make simple storing test results and visualisation of these in a BI dashboard☆45Updated last week
- Enforce Best Practices for all your Airflow DAGs. ⭐☆102Updated this week
- A provider package for kafka☆37Updated last year
- Trino dbt demo project to mix and load BigQuery data with and in a local PostgreSQL database☆75Updated 3 years ago
- Define, govern, and model event data for warehouse-first product analytics.☆83Updated 11 months ago
- Examples of various flow deployments for Prefect 1.0 (storage and run configurations)☆35Updated 3 years ago