chayansraj / Data-Pipeline-with-dbt-using-Airflow-on-GCPLinks
This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.
☆24Updated 3 months ago
Alternatives and similar repositories for Data-Pipeline-with-dbt-using-Airflow-on-GCP
Users that are interested in Data-Pipeline-with-dbt-using-Airflow-on-GCP are comparing it to the libraries listed below
Sorting:
- Cloned by the `dbt init` task☆62Updated last year
- A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset☆46Updated last year
- ☆21Updated last year
- Code for dbt tutorial☆165Updated 3 months ago
- Building a Data Pipeline with an Open Source Stack☆55Updated 5 months ago
- Data lake, data warehouse on GCP☆57Updated 3 years ago
- Full stack data engineering tools and infrastructure set-up☆57Updated 4 years ago
- build dw with dbt☆49Updated last year
- Execution of DBT models using Apache Airflow through Docker Compose☆126Updated 2 years ago
- End-to-end data platform leveraging the Modern data stack☆52Updated last year
- New generation opensource data stack☆76Updated 3 years ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆28Updated 3 years ago
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆91Updated 2 years ago
- ☆141Updated 10 months ago
- ☆92Updated 11 months ago
- ☆39Updated 9 months ago
- To provide a deeper understanding of how the modern, open-source data stack consisting of Iceberg, dbt, Trino, and Hive operates within a…☆44Updated last year
- Resources for video demonstrations and blog posts related to DataOps on AWS☆182Updated 3 years ago
- Data Engineering examples for Airflow, Prefect; dbt for BigQuery, Redshift, ClickHouse, Postgres, DuckDB; PySpark for Batch processing; K…☆68Updated last month
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.☆79Updated 2 years ago
- Installer for DataKitchen's Open Source Data Observability Products. Data breaks. Servers break. Your toolchain breaks. Ensure your team …☆129Updated last month
- End to end data engineering project☆57Updated 3 years ago
- Data engineering with dbt, published by Packt☆88Updated 3 months ago
- Cost Efficient Data Pipelines with DuckDB☆60Updated 7 months ago
- A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for …☆139Updated 5 years ago
- Simple stream processing pipeline☆110Updated last year
- Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python☆44Updated 2 years ago
- Building Data Lakehouse by open source technology. Support end to end data pipeline, from source data on AWS S3 to Lakehouse, visualize a…☆34Updated last year
- Capstone Project for DataExpert.io V4 Cohort☆11Updated last year
- Analytics engineering with dbt - projects and developer environment☆21Updated last year