skhatri / airflow-by-example
Bunch of Airflow Configurations and DAGs for Kubernetes, Spark based data-pipelines. Scale inside Kubernetes using spark kubernetes master. Secure it with keycloak
☆22Updated 2 years ago
Related projects: ⓘ
- A repository of sample code to show data quality checking best practices using Airflow.☆71Updated last year
- Learn how to add data validation and documentation to a data pipeline built with dbt and Airflow.☆167Updated 10 months ago
- The Python fake data producer for Apache Kafka® is a complete demo app allowing you to quickly produce JSON fake streaming datasets and …☆81Updated 4 months ago
- This repository is part of an article "Prefect workflow automation with Azure DevOps and AKS"☆31Updated 3 years ago
- Make simple storing test results and visualisation of these in a BI dashboard☆35Updated 2 weeks ago
- Astronomer Core Docker Images☆106Updated 3 months ago
- Pipeline definitions for managing data flows to power analytics at MIT Open Learning☆36Updated this week
- Pylint plugin for static code analysis on Airflow code☆89Updated 3 years ago
- Airflow Providers containing Deferrable Operators & Sensors from Astronomer☆135Updated this week
- ☆11Updated this week
- Great Expectations Airflow operator☆158Updated 2 weeks ago
- Code examples showing flow deployment to various types of infrastructure☆99Updated last year
- Pytest plugin for dbt core☆54Updated 3 months ago
- Airflow training for the crunch conf☆105Updated 5 years ago
- locopy: Loading/Unloading to Redshift and Snowflake using Python.☆104Updated this week
- Enforce Best Practices for all your Airflow DAGs. ⭐☆86Updated this week
- ☆20Updated 3 years ago
- New generation opensource data stack☆60Updated 2 years ago
- Trino dbt demo project to mix and load BigQuery data with and in a local PostgreSQL database☆64Updated 3 years ago
- Delta Lake Documentation☆45Updated 3 months ago
- ☆48Updated 2 years ago
- (project & tutorial) dag pipeline tests + ci/cd setup☆84Updated 3 years ago
- Utility functions for dbt projects running on Spark☆30Updated 10 months ago
- A provider package for kafka☆37Updated last year
- dbt + Trino demo project, using TPC-H sample data☆18Updated 5 months ago
- Collection of code snippets for blogs, conferences, and talks☆23Updated last year
- Full stack data engineering tools and infrastructure set-up☆38Updated 3 years ago
- Parse dbt artifacts and search dbt models with Algolia☆52Updated 3 years ago
- Sample Airflow DAGs☆60Updated last year
- A Python API for Asynchronously Loading Data into Snowflake DB -☆59Updated last week