codingvarun / streaming-elt-pipeline
This is a real-life, high throughput streaming ELT data pipeline for ecommerce
☆13Updated last year
Alternatives and similar repositories for streaming-elt-pipeline
Users that are interested in streaming-elt-pipeline are comparing it to the libraries listed below
Sorting:
- reating a modern data pipeline using a combination of Terraform, AWS Lambda and S3, Snowflake, DBT, Mage AI, and Dash.☆14Updated last year
- dbt / Amazon Redshift Demonstration Project☆34Updated 2 years ago
- This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS …☆19Updated 3 years ago
- DataHub on AWS demonstration resources☆10Updated 2 years ago
- Full stack data engineering tools and infrastructure set-up☆52Updated 4 years ago
- ☆17Updated 9 months ago
- A modern ELT demo using airbyte, dbt, snowflake and dagster☆27Updated 2 years ago
- Sample project that use Dagster, dbt, DuckDB and Dash to visualize car and motorcycle Spanish market☆58Updated 2 years ago
- Glue VSCode devcontainer setup☆14Updated 2 years ago
- Example Set up For DBT Cloud using Github Integrations☆11Updated 5 years ago
- Skeleton project for Apache Airflow training participants to work on.☆16Updated 4 years ago
- ☆18Updated 9 months ago
- Automate and streamline the alerting & notification process for dbt test results🐞🚀☆17Updated 3 weeks ago
- Snowflake & AWS Service Catalog Integration☆10Updated 2 years ago
- Utility functions for dbt projects running on Spark☆33Updated 3 months ago
- 📆 Run, schedule, and manage your dbt jobs using Kubernetes.☆24Updated 6 years ago
- Repo for holding the dbt project used to make sense of cloud cost data from the major cloud platforms☆37Updated 5 years ago
- Cost Efficient Data Pipelines with DuckDB☆52Updated 9 months ago
- Cloned by the `dbt init` task☆61Updated last year
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆55Updated 2 years ago
- CICD pipeline that deploys a dbt image on a GKE cluster☆11Updated 3 years ago
- Fully unit tested utility functions for data engineering. Python 3 only.☆16Updated 8 months ago
- Code to help generate SQL for stakeholders. Code at https://www.startdataengineering.com/post/data-democratize-llm/☆11Updated 11 months ago
- dlt-dagster-demo☆11Updated last year
- The goal of this project is to offer an AWS EMR template using Spot Fleet and On-Demand Instances that you can use quickly. Just focus on…☆27Updated 2 years ago
- Faker for Snowflake!☆33Updated 2 years ago
- dbt package for monitoring airflow DAGs and tasks☆29Updated 3 months ago
- An infrastructure as code approach to deploying Snowflake using Terraform☆25Updated last year
- duckdb-etl-framework☆10Updated 4 months ago
- ☆11Updated 5 months ago