chayansraj / Data-Pipeline-with-dbt-using-Airflow-on-GCP
This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.
☆23Updated this week
Alternatives and similar repositories for Data-Pipeline-with-dbt-using-Airflow-on-GCP:
Users that are interested in Data-Pipeline-with-dbt-using-Airflow-on-GCP are comparing it to the libraries listed below
- build dw with dbt☆44Updated 6 months ago
- End-to-end data platform leveraging the Modern data stack☆47Updated last year
- Cloned by the `dbt init` task☆61Updated 11 months ago
- Data lake, data warehouse on GCP☆56Updated 3 years ago
- Code snippets for Data Engineering Design Patterns book☆80Updated last month
- Execution of DBT models using Apache Airflow through Docker Compose☆116Updated 2 years ago
- Code for dbt tutorial☆156Updated 10 months ago
- 🥪🦘 An open source sandbox project exploring dbt workflows via a fictional sandwich shop's data.☆156Updated last week
- Data engineering with dbt, published by Packt☆77Updated last year
- ☆128Updated 2 months ago
- Data Engineering examples for Airflow, Prefect; dbt for BigQuery, Redshift, ClickHouse, Postgres, DuckDB; PySpark for Batch processing; K…☆65Updated 2 months ago
- Step-by-step tutorial on building a Kimball dimensional model with dbt☆137Updated 9 months ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆175Updated 3 years ago
- Data Engineering with Google Cloud Platform, published by Packt☆116Updated last year
- End to end data engineering project☆54Updated 2 years ago
- ⚙️ Airflow data pipeline with Terraform, GCP BigQuery, dbt, Soda and Looker Studio.☆21Updated last year
- Building a Data Pipeline with an Open Source Stack☆54Updated 9 months ago
- Delta-Lake, ETL, Spark, Airflow☆47Updated 2 years ago
- Local Environment to Practice Data Engineering☆142Updated 3 months ago
- A repository of sample code to show data quality checking best practices using Airflow.☆76Updated 2 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆36Updated 11 months ago
- Repository for Data Engineering Zoomcamp 2024☆14Updated last year
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆27Updated 2 years ago
- This project demonstrates how to build and automate an ETL pipeline written in Python and schedule it using open source Apache Airflow or…☆14Updated this week
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.☆67Updated last year
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆32Updated last year
- dbt / Amazon Redshift Demonstration Project☆34Updated 2 years ago
- Code for my "Efficient Data Processing in SQL" book.☆56Updated 8 months ago
- A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset☆41Updated 5 months ago
- Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python☆43Updated 2 years ago