jw-ng / airflow-dynamic-dags
☆13Updated 2 years ago
Alternatives and similar repositories for airflow-dynamic-dags:
Users that are interested in airflow-dynamic-dags are comparing it to the libraries listed below
- Full stack data engineering tools and infrastructure set-up☆47Updated 3 years ago
- ☆18Updated 5 months ago
- build dw with dbt☆33Updated 2 months ago
- PipeRider dbt workshop for DataTalksClub DE Zoomcamp☆17Updated last year
- Delta-Lake, ETL, Spark, Airflow☆45Updated 2 years ago
- ☆15Updated 11 months ago
- A repository of sample code to show data quality checking best practices using Airflow.☆74Updated last year
- Delta Lake Documentation☆48Updated 7 months ago
- ☆11Updated 2 years ago
- Data lake, data warehouse on GCP☆55Updated 3 years ago
- Code for my "Efficient Data Processing in SQL" book.☆54Updated 5 months ago
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆30Updated 10 months ago
- New generation opensource data stack☆65Updated 2 years ago
- Big Data Demystified meetup and blog examples☆31Updated 5 months ago
- Fake Pandas / PySpark DataFrame creator☆44Updated 10 months ago
- Sample Airflow DAGs to load data from the CovidTracking API to Snowflake via an AWS S3 intermediary.☆16Updated 4 years ago
- A project for exploring how Great Expectations can be used to ensure data quality and validate batches within a data pipeline defined in …☆21Updated 2 years ago
- This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and …☆27Updated 11 months ago
- Materials of the Official Helm Chart Webinar☆27Updated 3 years ago
- Cloned by the `dbt init` task☆60Updated 8 months ago
- This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging…☆75Updated 5 years ago
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆62Updated 3 months ago
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.☆62Updated last year
- Delta reader for the Ray open-source toolkit for building ML applications☆43Updated 11 months ago
- ☆40Updated 6 months ago
- A package to run DuckDB queries from Apache Airflow.☆15Updated 7 months ago
- Spark app to merge different schemas☆23Updated 4 years ago
- ☆14Updated last year
- Data-aware orchestration with dagster, dbt, and airbyte☆30Updated last year