This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgres and Streamlit. The system is built using Docker Compose to easily spin up the required services in Docker containers.
☆50Dec 11, 2023Updated 2 years ago
Alternatives and similar repositories for realtime-voting-data-engineering
Users that are interested in realtime-voting-data-engineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 3 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆12Nov 18, 2023Updated 2 years ago
- This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with …☆14Dec 27, 2023Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆47Jan 4, 2024Updated 2 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆44May 17, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- This project provides an end-to-end data processing and visualization of visa numbers in Japan using PySpark and Plotly. The spark cluste…☆11Oct 11, 2023Updated 3 years ago
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆25Jan 26, 2024Updated 2 years ago
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆40Dec 18, 2023Updated 2 years ago
- An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Az…☆31Oct 2, 2023Updated 3 years ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆47Mar 14, 2024Updated 2 years ago
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆145Jul 27, 2023Updated 3 years ago
- Modern Data Stack in a box with dbt-duckdb and Apache Superset☆16Updated this week
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆229Oct 23, 2023Updated 3 years ago
- ☆16Oct 8, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- This demo shows how to stream data to cloud databases with Confluent. It includes fully-managed connectors (Oracle CDC, RabbitMQ, MongoDB…☆11Jan 10, 2025Updated last year
- A demonstration of an ELT (Extract, Load, Transform) pipeline☆32Feb 19, 2024Updated 2 years ago
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆15Apr 29, 2021Updated 5 years ago
- Collection of quick starts on docker, terraform, ansible, etc☆18Apr 9, 2024Updated 2 years ago
- This project focuses on building a robust data pipeline using Apache Airflow to automate the ingestion of weather data from the OpenWeath…☆22Feb 3, 2026Updated 8 months ago
- Get Crypto data from API, stream it to Kafka with Airflow. Write data to MySQL and visualize with Metabase☆17Oct 2, 2023Updated 3 years ago
- SQLMesh example projects☆42Jul 2, 2025Updated last year
- CI/CD repository template to automate deployments of your production flows☆15Jul 1, 2024Updated 2 years ago
- Scan and monitor your network effortlessly! Nmap Prometheus Exporter provides insights into network health and security with Prometheus-c…☆15Oct 2, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Capstone Project for the IBM Data Engineering Professional Certification.☆13Mar 7, 2022Updated 4 years ago
- Data pipeline and meta-analysis dashboard for IT job postings from the web.☆11Jan 31, 2025Updated last year
- This project focuses on sentiment analysis of customer reviews on the Flipkart e-commerce platform. Leveraging natural language processin…☆18Nov 23, 2023Updated 2 years ago
- A hands-on, end-to-end cybersecurity SOC analyst training lab designed to simulate real-world scenarios in a safe environment. This 30-da…☆12Jun 27, 2025Updated last year
- A secure Python connector to the Kuda Microfinance Bank OpenAPI (v1.0.2)☆12Aug 13, 2022Updated 4 years ago
- This is a collecton of Amazon CDK projects to show how to directly ingest streaming data from Amazon Mananged Service for Apache Kafka (M…☆16Jul 28, 2026Updated 3 months ago
- Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake developme…☆12Feb 26, 2020Updated 6 years ago
- VIIRS Tutorial at Scipy 2021☆12Mar 29, 2025Updated last year
- ☆352Aug 13, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Resources and examples for creating Co-Pilot applications that use modern AI to assist with complex tasks.☆13Aug 27, 2025Updated last year
- ☆37Oct 28, 2024Updated 2 years ago
- ☆13Jun 14, 2024Updated 2 years ago
- ☆10Jul 19, 2020Updated 6 years ago
- ☆16Apr 18, 2025Updated last year
- A real-time streaming ETL pipeline for streaming and performing sentiment analysis on Twitter data using Apache Kafka, Apache Spark and D…☆29Aug 8, 2020Updated 6 years ago
- Code for "Efficient Data Processing in Spark" Course☆396Jun 29, 2026Updated 4 months ago