codingvarun / streaming-elt-pipeline
This is a real-life, high throughput streaming ELT data pipeline for ecommerce
☆13Updated last year
Alternatives and similar repositories for streaming-elt-pipeline:
Users that are interested in streaming-elt-pipeline are comparing it to the libraries listed below
- dbt (data build tool) projects targeting AWS analytics services (redshift, glue, emr, athena) and open table formats☆29Updated 2 years ago
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆14Updated last year
- Repo for holding the dbt project used to make sense of cloud cost data from the major cloud platforms☆37Updated 5 years ago
- dbt / Amazon Redshift Demonstration Project☆34Updated 2 years ago
- Example Set up For DBT Cloud using Github Integrations☆11Updated 5 years ago
- CICD pipeline that deploys a dbt image on a GKE cluster☆11Updated 3 years ago
- dbt package for monitoring airflow DAGs and tasks☆29Updated 2 months ago
- DataHub on AWS demonstration resources☆10Updated 2 years ago
- All the Snowflake Virtual Warehouse - Example☆12Updated 4 years ago
- Full stack data engineering tools and infrastructure set-up☆51Updated 4 years ago
- Glue VSCode devcontainer setup☆14Updated 2 years ago
- Utility functions for dbt projects running on Spark☆32Updated 2 months ago
- Build & Learn Data Engineering,Machine Learning over Kubernetes. No Shortcut approach.☆57Updated 2 years ago
- Collection of utility scripts to extract code so it can be upgraded to SnowFlake using the SnowConvert tool.☆14Updated this week
- Faker for Snowflake!☆33Updated 2 years ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆27Updated 2 years ago
- A template for dockerized dbt-Core projects with VS Code Dev Containers.☆21Updated 2 years ago
- Source code for the YouTube video, Apache Beam Explained in 12 Minutes☆21Updated 4 years ago
- ☆17Updated 8 months ago
- Sample code to collect Apache Iceberg metrics for table monitoring☆26Updated 8 months ago
- An example dbt project using AutomateDV to create a Data Vault 2.0 Data Warehouse based on the Snowflake TPC-H dataset.☆46Updated last year
- Code for my "Efficient Data Processing in SQL" book.☆56Updated 8 months ago
- Cloned by the `dbt init` task☆61Updated 11 months ago
- Creates simple data models on Snowflake to report dbt source freshness and tests☆26Updated last year
- Fully unit tested utility functions for data engineering. Python 3 only.☆16Updated 7 months ago
- Docker envinroment to stream data from Kafka to Iceberg tables☆26Updated last year
- Snowflake Guide: Building a Recommendation Engine Using Snowflake & Amazon SageMaker☆31Updated 3 years ago
- Yet Another (Spark) ETL Framework☆20Updated last year
- ☆17Updated 8 months ago
- Useful scripts, utilities, and tools for Snowflake☆13Updated 4 years ago