End-to-end ELT data engineering project
☆22Dec 24, 2022Updated 3 years ago
Alternatives and similar repositories for End-to-end-data-enginnerring-project
Users that are interested in End-to-end-data-enginnerring-project are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Building Data Warehouse on BigQuery which takes flat file as the data sources with Airflow as the Orchestrator☆12May 23, 2021Updated 4 years ago
- DataTalks.Club's Data Engineering Zoomcamp Project☆24Jul 14, 2022Updated 3 years ago
- In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related info…☆12Sep 9, 2023Updated 2 years ago
- A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apa…☆29Jun 7, 2023Updated 2 years ago
- A Data Engineering Project that implements an ETL data pipeline using Dagster, Apache Spark, Streamlit, MinIO, Metabase, Dbt, Polars, Doc…☆23Nov 19, 2024Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Scrape most mentioned stock tickers from Reddit. Wallstreetbets and Wallstreetbetsnew☆12Mar 5, 2021Updated 5 years ago
- Built a stream processing data pipeline to get data from disparate systems into a dashboard using Kafka as an intermediary.☆29Aug 14, 2023Updated 2 years ago
- Simple ETL pipeline using Python☆29May 22, 2023Updated 2 years ago
- Example Scala/SBT event producer for Amazon Kinesis☆21Mar 29, 2015Updated 11 years ago
- an end-to-end data pipeline extracting music listening habits and producing an insightful dashboard☆17Mar 31, 2024Updated 2 years ago
- Spark Structured Streaming data pipeline that processes movie ratings data in real-time.☆14Mar 1, 2026Updated last month
- 🌟 An end-to-end full-stack data science project, including modelling, MLOps, and data storytelling. ✨☆16Aug 30, 2025Updated 7 months ago
- Spark data pipeline that processes movie ratings data.☆31Apr 1, 2026Updated 2 weeks ago
- ☆13Aug 27, 2020Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A simple tool for monitoring the progress of OpenFOAM simulations☆13Nov 9, 2018Updated 7 years ago
- ODM (Object Document Mapper) for MongoDB based on python type hints with support for async and sync☆18Jan 24, 2026Updated 2 months ago
- Some functions to plot OpenFOAM data with Matplotlib☆11Apr 15, 2021Updated 5 years ago
- ☆16Sep 17, 2017Updated 8 years ago
- StarCraft 2 Data Pipeline with Airflow, DuckDB and Streamlit☆16Mar 14, 2024Updated 2 years ago
- ☆15Jan 26, 2023Updated 3 years ago
- Python wrapper for OpenFOAM meshes☆12Sep 16, 2025Updated 7 months ago
- ☆11Apr 9, 2022Updated 4 years ago
- Code for the Data Engineering Zoomcamp☆20Dec 12, 2022Updated 3 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Group Project: CFD solver taking heat into account, with transport of chemical substances and chemical reactions.☆12Oct 24, 2017Updated 8 years ago
- Demonstrating the concept of Google PubSub, a messaging queue service in Google, thru streaming fake financial data thru PubSub and query…☆15Aug 29, 2017Updated 8 years ago
- velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docke…☆20Aug 12, 2025Updated 8 months ago
- postProcessing tool for OpenFOAM, transform OpenFOAM fields to one single file by columns☆17May 11, 2021Updated 4 years ago
- Generate OpenAPI 3.x.x using Pydantic☆11Feb 9, 2023Updated 3 years ago
- [NeurIPS 2025@FoRLM] R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search☆17Jan 24, 2026Updated 2 months ago
- This guide will demonstrate how to deploy a minimal Apache Kafka cluster on Docker and set up producers and consumers using Python. We wi…☆18Nov 15, 2020Updated 5 years ago
- Code to build models that effectively predict promoter-driven gene expression☆12May 15, 2025Updated 11 months ago
- ☆12Oct 10, 2021Updated 4 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Fully dockerized Data Warehouse (DWH) using Airflow, dbt, PostgreSQL and dashboard using redash☆25Nov 12, 2022Updated 3 years ago
- Set up an async pipeline in python using Celery, RabbitMQ and MongoDB. This repo covers the end to end deployment of an async pipeline fo…☆13Sep 23, 2022Updated 3 years ago
- Data and code for ACL 2023 paper "RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations"☆15Feb 8, 2024Updated 2 years ago
- Google Ad Manager API Client Library for NodeJs.☆12Jul 2, 2023Updated 2 years ago
- Coupon System project: SpringBoot & AngularTS☆12Jan 3, 2021Updated 5 years ago
- Example using Great Expectations to Validate Data in a scikit-learn Pipeline☆21Jul 23, 2020Updated 5 years ago
- Demo of structured, contextual JSON logging with Spring Boot and Log4j2☆15Feb 15, 2022Updated 4 years ago