airscholar / realtime-voting-data-engineeringLinks
This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgres and Streamlit. The system is built using Docker Compose to easily spin up the required services in Docker containers.
☆41Updated last year
Alternatives and similar repositories for realtime-voting-data-engineering
Users that are interested in realtime-voting-data-engineering are comparing it to the libraries listed below
Sorting:
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆33Updated last year
- DataTalks.Club's Data Engineering Zoomcamp Project☆23Updated 3 years ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆146Updated last year
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆38Updated last year
- YouTube tutorial project☆106Updated last year
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆265Updated 5 months ago
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆98Updated 3 months ago
- End to end data engineering project☆57Updated 2 years ago
- ☆142Updated 2 years ago
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆160Updated 2 years ago
- ☆201Updated last year
- This project shows how to capture changes from postgres database and stream them into kafka☆36Updated last year
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆139Updated last year
- Stream processing pipeline from Finnhub websocket using Spark, Kafka, Kubernetes and more☆352Updated last year
- Ultimate guide for mastering Spark Performance Tuning and Optimization concepts and for preparing for Data Engineering interviews☆152Updated last year
- ☆151Updated 3 years ago
- Data Engineering YouTube Analysis Project by Darshil Parmar☆197Updated last year
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆147Updated 5 years ago
- Price Crawler - Tracking Price Inflation☆185Updated 5 years ago
- Simple ETL pipeline using Python☆26Updated 2 years ago
- A template repository to create a data project with IAC, CI/CD, Data migrations, & testing☆266Updated last year
- ☆28Updated last year
- This repo is for the Linkedin Learning course: End-to-End Data Engineering Project☆23Updated last year
- Sample project to demonstrate data engineering best practices☆194Updated last year
- Simple stream processing pipeline☆103Updated last year
- ☆67Updated last month
- With everything I learned from DEZoomcamp from datatalks.club, this project performs a batch processing on AWS for the cycling dataset wh…☆14Updated 3 years ago
- Near real time ETL to populate a dashboard.☆72Updated last year
- ☆282Updated 11 months ago
- This project contain build end-to-end e-commerce data from data source into data warehouse and visualization.☆13Updated 10 months ago