codingvarun / streaming-elt-pipeline
This is a real-life, high throughput streaming ELT data pipeline for ecommerce
☆13Updated last year
Alternatives and similar repositories for streaming-elt-pipeline:
Users that are interested in streaming-elt-pipeline are comparing it to the libraries listed below
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆14Updated last year
- All the Snowflake Virtual Warehouse - Example☆11Updated 4 years ago
- dbt / Amazon Redshift Demonstration Project☆34Updated 2 years ago
- Full stack data engineering tools and infrastructure set-up☆49Updated 4 years ago
- The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on…☆26Updated 2 years ago
- ☆15Updated 6 months ago
- Automate and streamline the alerting & notification process for dbt test results🐞🚀☆17Updated last week
- A repo to track data engineering projects☆13Updated 2 years ago
- Useful scripts, utilities, and tools for Snowflake☆13Updated 4 years ago
- DataHub on AWS demonstration resources☆10Updated 2 years ago
- ☆17Updated 6 months ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆26Updated 2 years ago
- Sample code to collect Apache Iceberg metrics for table monitoring☆24Updated 6 months ago
- Fully unit tested utility functions for data engineering. Python 3 only.☆15Updated 5 months ago
- a pytest plugin for dbt adapter test suites☆19Updated last year
- Collection of utility scripts to extract code so it can be upgraded to SnowFlake using the SnowConvert tool.☆12Updated this week
- Snowflake Guide: Building a Recommendation Engine Using Snowflake & Amazon SageMaker☆31Updated 3 years ago
- CICD pipeline that deploys a dbt image on a GKE cluster☆11Updated 3 years ago
- Code snippets for Data Engineering Design Patterns book☆69Updated 2 weeks ago
- Cost Efficient Data Pipelines with DuckDB☆49Updated 6 months ago
- Utility functions for dbt projects running on Spark☆31Updated last week
- DBT Package reproducing dbt incremental materialization leveraging on Snowflake streams☆28Updated 3 months ago
- Repo for holding the dbt project used to make sense of cloud cost data from the major cloud platforms☆37Updated 4 years ago
- ☆41Updated 7 months ago
- Glue VSCode devcontainer setup☆14Updated 2 years ago
- Creates simple data models on Snowflake to report dbt source freshness and tests☆23Updated last year
- Code for my "Efficient Data Processing in SQL" book.☆56Updated 6 months ago
- Deployed an kafka instance in AWS EC2 Instance to streamline the data into Databricks☆10Updated last year
- A template for dockerized dbt-Core projects with VS Code Dev Containers.☆20Updated 2 years ago