airscholar / realtime-voting-data-engineering
This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgres and Streamlit. The system is built using Docker Compose to easily spin up the required services in Docker containers.
☆35Updated last year
Alternatives and similar repositories for realtime-voting-data-engineering:
Users that are interested in realtime-voting-data-engineering are comparing it to the libraries listed below
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆27Updated last year
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆86Updated 7 months ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆34Updated last year
- ☆27Updated last year
- DataTalks.Club's Data Engineering Zoomcamp Project☆23Updated 2 years ago
- ☆40Updated 8 months ago
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆11Updated last year
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆15Updated 3 years ago
- End to end data engineering project☆53Updated 2 years ago
- YouTube tutorial project☆100Updated last year
- Classwork projects and home works done through Udacity data engineering nano degree☆74Updated last year
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆124Updated last year
- Writes the CSV file to Postgres, read table and modify it. Write more tables to Postgres with Airflow.☆35Updated last year
- 😈Complete End to End ETL Pipeline with Spark, Airflow, & AWS☆44Updated 5 years ago
- ☆135Updated 2 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆36Updated 10 months ago
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆147Updated 2 years ago
- Resources and projects from Udacity Data Engineering with AWS nano degree programme☆25Updated last year
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆141Updated 4 years ago
- ☆87Updated 2 years ago
- ☆151Updated 2 years ago
- Simple ETL pipeline using Python☆25Updated last year
- Recohut - Learn data engineering, data science☆96Updated last year
- Welcome to my data engineering projects repository! Here you will find a collection of data engineering projects that I have worked on.☆17Updated last year
- Solution to all projects of Udacity's Data Engineering Nanodegree: Data Modeling with Postgres & Cassandra, Data Warehouse with Redshift,…☆56Updated 2 years ago
- In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…☆21Updated last year
- A project portfolio to accompany my resume☆27Updated last year
- ☆65Updated this week
- This project focuses on building a robust data pipeline using Apache Airflow to automate the ingestion of weather data from the OpenWeath…☆21Updated last year
- This project contain build end-to-end e-commerce data from data source into data warehouse and visualization.☆12Updated 6 months ago