aymane-maghouti / Real-Time-Streaming-Kafka-Debezium-Spark-StreamingLinks
This project demonstrates real-time data streaming and processing architecture using Kafka, Spark Streaming, and Debezium for capturing CDC (Change Data Capture) events. The pipeline collects transaction data, processes it in real time, and updates a dashboard to display real-time analytics for smartphone data.
☆13Updated last year
Alternatives and similar repositories for Real-Time-Streaming-Kafka-Debezium-Spark-Streaming
Users that are interested in Real-Time-Streaming-Kafka-Debezium-Spark-Streaming are comparing it to the libraries listed below
Sorting:
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆45Updated 2 years ago
- This project shows how to capture changes from postgres database and stream them into kafka☆40Updated last year
- In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our …☆38Updated 2 years ago
- ☆70Updated this week
- ☆21Updated 2 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆44Updated 2 years ago
- End to end data engineering project with kafka, airflow, spark, postgres and docker.☆107Updated 3 weeks ago
- ☆19Updated 2 years ago
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆47Updated 2 years ago
- Resources and projects from Udacity Data Engineering with AWS nano degree programme☆28Updated 2 years ago
- End to end data engineering project☆58Updated 3 years ago
- This project demonstrates how to use Apache Airflow to submit jobs to Apache spark cluster in different programming laguages using Python…☆46Updated last year
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆310Updated 11 months ago
- ☆44Updated 5 years ago
- Simple stream processing pipeline☆110Updated last year
- With everything I learned from DEZoomcamp from datatalks.club, this project performs a batch processing on AWS for the cycling dataset wh…☆15Updated 3 weeks ago
- Public Docker Images for popular services☆45Updated 4 months ago
- Data Engineering with AWS, 2nd edition - Published by Packt☆168Updated 2 years ago
- Nyc_Taxi_Data_Pipeline - DE Project☆133Updated last year
- Sample repo for startdataengineering DE 101 free course☆74Updated last year
- This project provides a comprehensive data pipeline solution to extract, transform, and load (ETL) Reddit data into a Redshift data wareh…☆203Updated 2 years ago
- ☆47Updated 4 years ago
- Contains spark dataframe solutions of leetcode questions☆26Updated 3 years ago
- Apartments Data Pipeline using Airflow and Spark.☆23Updated 3 years ago
- ☆88Updated 3 years ago
- data-warehouse-snowflake-for-data-engineering☆19Updated 2 years ago
- YouTube tutorial project☆107Updated 2 years ago
- Projects done in the Data Engineer Nanodegree Program by Udacity.com☆164Updated 3 years ago
- Cool DE Projects☆52Updated last month
- In this project, we will build and ETL(Extract,Transform,Load) pipeline using the Spotify API on AWS. The pipeline will retrieve data fro…☆25Updated 2 years ago