chayansraj / Data-Pipeline-with-dbt-using-Airflow-on-GCP
This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.
☆20Updated last week
Alternatives and similar repositories for Data-Pipeline-with-dbt-using-Airflow-on-GCP:
Users that are interested in Data-Pipeline-with-dbt-using-Airflow-on-GCP are comparing it to the libraries listed below
- Cloned by the `dbt init` task☆60Updated 9 months ago
- Code for dbt tutorial☆149Updated 8 months ago
- End to end data engineering project☆53Updated 2 years ago
- Data engineering with dbt, published by Packt☆66Updated 10 months ago
- Data lake, data warehouse on GCP☆55Updated 3 years ago
- ☆117Updated 3 months ago
- build dw with dbt☆35Updated 3 months ago
- ☆23Updated 2 weeks ago
- ☆14Updated last year
- ☆106Updated 6 months ago
- Containerized end-to-end analytics of Spotify data using Python, dbt, Postgres, and Metabase☆124Updated 2 years ago
- Analytics engineering with dbt - projects and developer environment☆16Updated 4 months ago
- Execution of DBT models using Apache Airflow through Docker Compose☆113Updated 2 years ago
- ☆73Updated 3 months ago
- Covid19 and Iowa Liquor Sales analysis at BigQuery using dbt, Airflow, Marquez, Google Cloud and other modern data stack tools☆14Updated 2 years ago
- Sample project to demonstrate data engineering best practices☆175Updated 11 months ago
- End-to-end data platform leveraging the Modern data stack☆45Updated 9 months ago
- Simple stream processing pipeline☆96Updated 7 months ago
- 🥪🦘 An open source sandbox project exploring dbt workflows via a fictional sandwich shop's data.☆134Updated this week
- A repository of sample code to show data quality checking best practices using Airflow.☆74Updated last year
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆135Updated 4 years ago
- Resources for video demonstrations and blog posts related to DataOps on AWS☆173Updated 3 years ago
- A tutorial for the Great Expectations library.☆69Updated 3 years ago
- A template repository to create a data project with IAC, CI/CD, Data migrations, & testing☆254Updated 6 months ago
- Code for "Advanced data transformations in SQL" free live workshop☆71Updated 3 months ago
- A data engineering project with Airflow, dbt, Terrafrom, GCP and much more!☆22Updated 2 years ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆25Updated last year
- A Python package that creates fine-grained dbt tasks on Apache Airflow☆63Updated 4 months ago
- Code for my "Efficient Data Processing in SQL" book.☆56Updated 5 months ago
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.☆62Updated last year