This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgres and Streamlit. The system is built using Docker Compose to easily spin up the required services in Docker containers.
☆51Dec 11, 2023Updated 2 years ago
Alternatives and similar repositories for realtime-voting-data-engineering
Users that are interested in realtime-voting-data-engineering are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An end-to-end data engineering pipeline that fetches real-time YouTube analytics and streams them through Kafka for processing with ksqlD…☆16Sep 19, 2023Updated 3 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆12Nov 18, 2023Updated 3 years ago
- This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with …☆14Dec 27, 2023Updated 2 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆44May 17, 2024Updated 2 years ago
- This repository contains the necessary configuration files and DAGs (Directed Acyclic Graphs) for setting up a robust data engineering en…☆25Jan 26, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- This project provides an end-to-end data processing and visualization of visa numbers in Japan using PySpark and Plotly. The spark cluste…☆11Oct 11, 2023Updated 3 years ago
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆381Feb 14, 2025Updated last year
- End-to-end data pipeline that ingests, processes, and stores data. It uses Apache Airflow to schedule scripts that fetch data from an API…☆23Sep 23, 2026Updated last month
- Apache Airflow advanced functionalities examples☆21Mar 22, 2024Updated 2 years ago
- Fully dockerized Data Warehouse (DWH) using Airflow, dbt, PostgreSQL and dashboard using redash☆27Nov 12, 2022Updated 4 years ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆47Mar 14, 2024Updated 2 years ago
- Modern Data Stack in a box with dbt-duckdb and Apache Superset☆16Nov 12, 2026Updated last week
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆145Jul 27, 2023Updated 3 years ago
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆230Oct 23, 2023Updated 3 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Loan Default Prediction using PySpark, with jobs scheduled by Apache Airflow and Integration with Spark using Apache Livy☆22Dec 26, 2020Updated 5 years ago
- capstone project for Dataengineer.io bootcamp Public Repo☆12Feb 20, 2024Updated 2 years ago
- Create Interactive Dashboards With Streamlit in Python☆19Aug 12, 2020Updated 6 years ago
- A simple pipeline utilising cron, Postgres, AWS EC2, and Metabase☆13Jul 9, 2024Updated 2 years ago
- This demo shows how to stream data to cloud databases with Confluent. It includes fully-managed connectors (Oracle CDC, RabbitMQ, MongoDB…☆11Jan 10, 2025Updated last year
- A demonstration of an ELT (Extract, Load, Transform) pipeline☆32Feb 19, 2024Updated 2 years ago
- A collection of data engineering projects: data modeling, ETL pipelines, data lakes, infrastructure configuration on AWS, data warehousin…☆15Apr 29, 2021Updated 5 years ago
- The Modern Data Stack in a (Smaller) Box☆12Jan 28, 2023Updated 3 years ago
- A streamlit multipage app template for geospatial applications☆66Apr 26, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Collection of quick starts on docker, terraform, ansible, etc☆18Apr 9, 2024Updated 2 years ago
- ☆11Jun 16, 2019Updated 7 years ago
- SQLMesh example projects☆42Jul 2, 2025Updated last year
- A dockerized 5 min. mini-batch data pipeline. Spark Structured Streaming: reading from Kafka to PostgreSQL as a sink DB. Airflow, Kafka, …☆10Jul 17, 2024Updated 2 years ago
- Get Crypto data from API, stream it to Kafka with Airflow. Write data to MySQL and visualize with Metabase☆17Oct 2, 2023Updated 3 years ago
- Create a Kubernetes Cluster using Ansible on Contabo Unbuntu 22.04 VPS servers.☆10Jul 6, 2024Updated 2 years ago
- CI/CD repository template to automate deployments of your production flows☆15Jul 1, 2024Updated 2 years ago
- Implementation of a Q&A Chatbot with RAG using the Data Engineering Community WhatsApp group messages.☆10Jan 17, 2025Updated last year
- Capstone Project for the IBM Data Engineering Professional Certification.☆13Mar 7, 2022Updated 4 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆122Jan 8, 2026Updated 10 months ago
- Data pipeline and meta-analysis dashboard for IT job postings from the web.☆11Jan 31, 2025Updated last year
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆66Jul 21, 2023Updated 3 years ago
- Welcome to Power BI Embedded Step by Step Series. Using this GitHub Repository you can download complete solution.☆16Jan 2, 2021Updated 5 years ago
- ☆12Oct 15, 2021Updated 5 years ago
- Set up a Cost-Effective Modern Data Stack for a Charity☆20Mar 26, 2025Updated last year
- A Coronavirus Dashboard that updates information realtime using Streamlit☆18Oct 3, 2023Updated 3 years ago
Are these results useful?