chayansraj / Data-Pipeline-with-dbt-using-Airflow-on-GCPLinks
This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.
☆24Updated last week
Alternatives and similar repositories for Data-Pipeline-with-dbt-using-Airflow-on-GCP
Users that are interested in Data-Pipeline-with-dbt-using-Airflow-on-GCP are comparing it to the libraries listed below
Sorting:
- Cloned by the `dbt init` task☆60Updated last year
- Code for dbt tutorial☆156Updated last month
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.☆74Updated last year
- Data lake, data warehouse on GCP☆56Updated 3 years ago
- build dw with dbt☆47Updated 8 months ago
- ☆14Updated 2 years ago
- Open Data Stack Projects: Examples of End to End Data Engineering Projects☆86Updated 2 years ago
- A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, PostgreSQL and Superset☆45Updated 8 months ago
- End-to-end data platform leveraging the Modern data stack☆50Updated last year
- ☆41Updated last year
- Code for blog at https://www.startdataengineering.com/post/python-for-de/☆80Updated last year
- New generation opensource data stack☆70Updated 3 years ago
- ☆202Updated 5 months ago
- Simple stream processing pipeline☆103Updated last year
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆28Updated 2 years ago
- Data engineering with dbt, published by Packt☆81Updated last year
- Code snippets for Data Engineering Design Patterns book☆128Updated 3 months ago
- ☆134Updated 5 months ago
- ⚙️ Airflow data pipeline with Terraform, GCP BigQuery, dbt, Soda and Looker Studio.☆21Updated last year
- Building a Data Pipeline with an Open Source Stack☆55Updated 2 weeks ago
- End to end data engineering project☆57Updated 2 years ago
- Code for blog at: https://www.startdataengineering.com/post/docker-for-de/☆38Updated last year
- Execution of DBT models using Apache Airflow through Docker Compose☆117Updated 2 years ago
- Full stack data engineering tools and infrastructure set-up☆53Updated 4 years ago
- Analytics engineering with dbt - projects and developer environment☆19Updated 9 months ago
- Sample project to demonstrate data engineering best practices☆194Updated last year
- Data Engineering examples for Airflow, Prefect; dbt for BigQuery, Redshift, ClickHouse, Postgres, DuckDB; PySpark for Batch processing; K…☆66Updated 3 weeks ago
- ☆37Updated 4 months ago
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆42Updated last year
- Resources for video demonstrations and blog posts related to DataOps on AWS☆178Updated 3 years ago