codingvarun / streaming-elt-pipeline
This is a real-life, high throughput streaming ELT data pipeline for ecommerce
☆13Updated last year
Related projects ⓘ
Alternatives and complementary repositories for streaming-elt-pipeline
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆12Updated last year
- Full stack data engineering tools and infrastructure set-up☆44Updated 3 years ago
- ☆15Updated 3 months ago
- All the Snowflake Virtual Warehouse - Example☆11Updated 4 years ago
- Build & Learn Data Engineering,Machine Learning over Kubernetes. No Shortcut approach.☆58Updated last year
- The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on…☆26Updated 2 years ago
- A repo to track data engineering projects☆13Updated 2 years ago
- Snowflake Guide: Building a Recommendation Engine Using Snowflake & Amazon SageMaker☆31Updated 3 years ago
- CI/CD for Snowflake using Jenkins and Sqitch☆8Updated 5 years ago
- Example Set up For DBT Cloud using Github Integrations☆11Updated 4 years ago
- End-to-End ELT data pipeline with Postgres, Airbyte, dbt, Dagster, Snowflake and Metabase☆10Updated last year
- dbt / Amazon Redshift Demonstration Project☆33Updated last year
- GitHub repository related to the course Mastering Elastic Map Reduce for Data Engineers☆23Updated 2 years ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆24Updated last year
- CICD pipeline that deploys a dbt image on a GKE cluster☆11Updated 3 years ago
- ☆16Updated 3 months ago
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆14Updated 3 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- Cloned by the `dbt init` task☆59Updated 6 months ago
- Analytics engineering with dbt - projects and developer environment☆16Updated last month
- ☆60Updated last week
- This construct builds some elements for you to quickly launch an EMR Serverless application. After submitting the Emr Serverless job, you…☆10Updated this week
- Code for my "Efficient Data Processing in SQL" book.☆50Updated 3 months ago
- Repo for holding the dbt project used to make sense of cloud cost data from the major cloud platforms☆37Updated 4 years ago
- dbt package for monitoring airflow DAGs and tasks☆29Updated this week
- A data engineering project with Airflow, dbt, Terrafrom, GCP and much more!☆22Updated 2 years ago
- A custom end-to-end data pipeline for customer churn☆9Updated 3 weeks ago
- An example dbt project using AutomateDV to create a Data Vault 2.0 Data Warehouse based on the Snowflake TPC-H dataset.☆41Updated 7 months ago
- ☆38Updated 4 months ago
- This is a demo streaming project simulating a music streaming service.☆27Updated 3 months ago