jaumpedro214 / traffic-flow-spark-kafkaLinks
Testing Spark Structured Streaming anf Kafka with real data from traffic sensors
☆16Updated 2 years ago
Alternatives and similar repositories for traffic-flow-spark-kafka
Users that are interested in traffic-flow-spark-kafka are comparing it to the libraries listed below
Sorting:
- ☆12Updated 4 years ago
- This project aims to build a streaming application to perform real-time analytics of Covid-19 related tweets and deploy an ML model for r…☆14Updated 4 years ago
- ☆44Updated last year
- ☆37Updated 5 years ago
- Design/Implement stream/batch architecture on NYC taxi data | #DE☆25Updated 4 years ago
- Project for real-time anomaly detection using Kafka and python☆58Updated 2 years ago
- Mastering Big Data Analytics with PySpark, Published by Packt☆161Updated last year
- Project for "Data pipeline design patterns" blog.☆46Updated last year
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆42Updated 2 years ago
- Delta-Lake, ETL, Spark, Airflow☆48Updated 2 years ago
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO☆63Updated 2 years ago
- A batch processing data pipeline, using AWS resources (S3, EMR, Redshift, EC2, IAM), provisioned via Terraform, and orchestrated from loc…☆23Updated 3 years ago
- ☆40Updated 2 years ago
- End to end data engineering project☆57Updated 2 years ago
- Base Kafka Producer, consumer, flask api and PySpark Structured streaming Job☆11Updated 3 years ago
- This repository contains an Apache Flink application for real-time sales analytics built using Docker Compose to orchestrate the necessar…☆44Updated last year
- Deploying Models to Production with Mlflow and AWS Sagemaker☆23Updated 4 years ago
- PySpark functions and utilities with examples. Assists ETL process of data modeling☆104Updated 4 years ago
- Course Material Data Engineering on AWS Course☆29Updated last year
- A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract d…☆24Updated 3 years ago
- A list of all my posts and personal projects☆74Updated last year
- Simple stream processing pipeline☆108Updated last year
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandra☆143Updated 2 years ago
- Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)☆318Updated last year
- Template for data pipelines, ML workflows, API dev and monitoring☆45Updated last year
- ☆40Updated 2 years ago
- ☆68Updated last month
- Code for dbt tutorial☆162Updated 2 weeks ago
- Apache Spark using SQL☆14Updated 4 years ago
- This repository contains the code for a realtime election voting system. The system is built using Python, Kafka, Spark Streaming, Postgr…☆41Updated last year