skhatri / airflow-by-example
Bunch of Airflow Configurations and DAGs for Kubernetes, Spark based data-pipelines. Scale inside Kubernetes using spark kubernetes master. Secure it with keycloak
☆23Updated 3 years ago
Alternatives and similar repositories for airflow-by-example:
Users that are interested in airflow-by-example are comparing it to the libraries listed below
- Creates simple data models on Snowflake to report dbt source freshness and tests☆25Updated last year
- A repository of sample code to show data quality checking best practices using Airflow.☆74Updated 2 years ago
- This repository is part of an article "Prefect workflow automation with Azure DevOps and AKS"☆31Updated 4 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆86Updated 4 years ago
- Make simple storing test results and visualisation of these in a BI dashboard☆43Updated last week
- Utility functions for dbt projects running on Spark☆31Updated last month
- ☆50Updated 4 years ago
- Great Expectations Airflow operator☆161Updated this week
- Airflow training for the crunch conf☆105Updated 6 years ago
- Pipeline definitions for managing data flows to power analytics at MIT Open Learning☆43Updated this week
- Astronomer Core Docker Images☆106Updated 10 months ago
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆167Updated last year
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆65Updated 6 months ago
- Parse dbt artifacts and search dbt models with Algolia☆52Updated 3 years ago
- dbt + Trino demo project, using TPC-H sample data☆19Updated last year
- Airflow Providers containing Deferrable Operators & Sensors from Astronomer☆146Updated this week
- Pylint plugin for static code analysis on Airflow code☆93Updated 4 years ago
- ☆20Updated 3 years ago
- Full stack data engineering tools and infrastructure set-up☆50Updated 4 years ago
- Code snippets for Data Engineering Design Patterns book☆75Updated last week
- Pytest plugin for dbt core☆59Updated 2 months ago
- Sample Airflow DAGs☆62Updated 2 years ago
- ☆49Updated 3 years ago
- Library to convert DBT manifest metadata to Airflow tasks☆48Updated last year
- Updated 6 months ago
- Dask integration for Snowflake☆30Updated 4 months ago
- A simple and easy to use Data Quality (DQ) tool built with Python.☆50Updated last year
- Repo for orienting dbt users to the Dagster asset framework☆54Updated 2 years ago
- Trino dbt demo project to mix and load BigQuery data with and in a local PostgreSQL database☆73Updated 3 years ago
- how to unit test your PySpark code☆28Updated 4 years ago