skhatri / airflow-by-example
Bunch of Airflow Configurations and DAGs for Kubernetes, Spark based data-pipelines. Scale inside Kubernetes using spark kubernetes master. Secure it with keycloak
☆23Updated 3 years ago
Alternatives and similar repositories for airflow-by-example:
Users that are interested in airflow-by-example are comparing it to the libraries listed below
- A repository of sample code to show data quality checking best practices using Airflow.☆76Updated 2 years ago
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆168Updated last year
- Make simple storing test results and visualisation of these in a BI dashboard☆43Updated last month
- Enforce Best Practices for all your Airflow DAGs. ⭐☆99Updated this week
- Airflow Providers containing Deferrable Operators & Sensors from Astronomer☆147Updated this week
- Pipeline definitions for managing data flows to power analytics at MIT Open Learning☆43Updated this week
- ☆49Updated 3 years ago
- Cloned by the `dbt init` task☆61Updated 11 months ago
- Astronomer Core Docker Images☆107Updated 11 months ago
- Pytest plugin for dbt core☆60Updated 3 months ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆87Updated 4 years ago
- Great Expectations Airflow operator☆163Updated this week
- Code examples showing flow deployment to various types of infrastructure☆106Updated 2 years ago
- Airflow training for the crunch conf☆105Updated 6 years ago
- Creates simple data models on Snowflake to report dbt source freshness and tests☆26Updated last year
- A simple and easy to use Data Quality (DQ) tool built with Python.☆50Updated last year
- A curated list of dagster code snippets for data engineers☆54Updated last year
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆27Updated 2 years ago
- Updated 7 months ago
- Companion repository for the "Streamlining AWS Glue CI/CD — A Comprehensive Blueprint" blog post☆12Updated 5 months ago
- Pylint plugin for static code analysis on Airflow code☆93Updated 4 years ago
- Delta Lake helper methods. No Spark dependency.☆23Updated 7 months ago
- Trino dbt demo project to mix and load BigQuery data with and in a local PostgreSQL database☆74Updated 3 years ago
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆68Updated 7 months ago
- ☆199Updated last year
- New generation opensource data stack☆67Updated 2 years ago
- Dry run capability for dbt projects using BigQuery☆96Updated 2 weeks ago
- Fast iterative local development and testing of Apache Airflow workflows☆200Updated last week
- Visualize dependencies between Airflow DAGs☆49Updated 3 years ago
- Read Delta tables without any Spark☆47Updated last year