codingvarun / streaming-elt-pipeline
This is a real-life, high throughput streaming ELT data pipeline for ecommerce
☆13Updated last year
Alternatives and similar repositories for streaming-elt-pipeline:
Users that are interested in streaming-elt-pipeline are comparing it to the libraries listed below
- The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on…☆26Updated 2 years ago
- All the Snowflake Virtual Warehouse - Example☆11Updated 4 years ago
- Fully unit tested utility functions for data engineering. Python 3 only.☆15Updated 4 months ago
- Full stack data engineering tools and infrastructure set-up☆47Updated 3 years ago
- CICD pipeline that deploys a dbt image on a GKE cluster☆11Updated 3 years ago
- ☆18Updated 5 months ago
- Apache Airflow advanced functionalities examples☆13Updated 9 months ago
- dbt / Amazon Redshift Demonstration Project☆33Updated 2 years ago
- Snowflake Guide: Building a Recommendation Engine Using Snowflake & Amazon SageMaker☆31Updated 3 years ago
- Cloned by the `dbt init` task☆60Updated 8 months ago
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆13Updated last year
- DataHub on AWS demonstration resources☆10Updated last year
- Collection of utility scripts to extract code so it can be upgraded to SnowFlake using the SnowConvert tool.☆12Updated last month
- Build & Learn Data Engineering,Machine Learning over Kubernetes. No Shortcut approach.☆58Updated 2 years ago
- PySpark Cheatsheet☆35Updated 2 years ago
- This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS …☆19Updated 3 years ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆25Updated 2 years ago
- ☆61Updated last week
- dbt package for monitoring airflow DAGs and tasks☆29Updated this week
- An example dbt project using AutomateDV to create a Data Vault 2.0 Data Warehouse based on the Snowflake TPC-H dataset.☆45Updated 9 months ago
- Automate and streamline the alerting & notification process for dbt test results🐞🚀☆17Updated this week
- Creates simple data models on Snowflake to report dbt source freshness and tests☆23Updated last year
- A repo to track data engineering projects☆13Updated 2 years ago
- Guide for running a custom API Powered by Snowflake in Python☆20Updated 5 months ago
- ☆11Updated 2 years ago
- GitHub repository related to the course Mastering Elastic Map Reduce for Data Engineers☆24Updated 2 years ago
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆14Updated 3 years ago
- Data-aware orchestration with dagster, dbt, and airbyte☆30Updated last year
- ☆49Updated 9 months ago
- Code to help generate SQL for stakeholders. Code at https://www.startdataengineering.com/post/data-democratize-llm/☆12Updated 7 months ago