chayansraj / Data-Pipeline-with-dbt-using-Airflow-on-GCPLinks
This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.
☆24Updated 3 months ago
Alternatives and similar repositories for Data-Pipeline-with-dbt-using-Airflow-on-GCP
Users that are interested in Data-Pipeline-with-dbt-using-Airflow-on-GCP are comparing it to the libraries listed below
Sorting:
- Cloned by the `dbt init` task☆62Updated last year
- Code for dbt tutorial☆165Updated 2 months ago
- ☆21Updated last year
- A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset☆46Updated last year
- ☆141Updated 9 months ago
- ☆15Updated 2 years ago
- Data engineering with dbt, published by Packt☆87Updated 2 months ago
- ☆92Updated 10 months ago
- Cost Efficient Data Pipelines with DuckDB☆60Updated 6 months ago
- New generation opensource data stack☆75Updated 3 years ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆28Updated 3 years ago
- The Ultimate Guide to Snowpark, published by Packt☆15Updated last year
- Resources for video demonstrations and blog posts related to DataOps on AWS☆182Updated 3 years ago
- Data lake, data warehouse on GCP☆57Updated 3 years ago
- Simple stream processing pipeline☆110Updated last year
- Analytics engineering with dbt - projects and developer environment☆21Updated last year
- Code snippets for Data Engineering Design Patterns book☆275Updated 8 months ago
- Full stack data engineering tools and infrastructure set-up☆57Updated 4 years ago
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆42Updated last year
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆91Updated 2 years ago
- A simple Data Engineering solution for testing or education purposes. You only need to know SQL and Python to understand this project. Da…☆27Updated 3 years ago
- ⚙️ Airflow data pipeline with Terraform, GCP BigQuery, dbt, Soda and Looker Studio.☆22Updated 2 years ago
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆22Updated last year
- End to end data engineering project☆57Updated 3 years ago
- Execution of DBT models using Apache Airflow through Docker Compose☆125Updated 2 years ago
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.☆79Updated 2 years ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆36Updated last year
- Dagster University courses☆116Updated this week
- build dw with dbt☆49Updated last year
- Code for "Advanced data transformations in SQL" free live workshop☆87Updated 6 months ago