jaumpedro214 / traffic-flow-spark-kafkaLinks
Testing Spark Structured Streaming anf Kafka with real data from traffic sensors
☆16Updated 2 years ago
Alternatives and similar repositories for traffic-flow-spark-kafka
Users that are interested in traffic-flow-spark-kafka are comparing it to the libraries listed below
Sorting:
- ☆37Updated 5 years ago
- This project aims to build a streaming application to perform real-time analytics of Covid-19 related tweets and deploy an ML model for r…☆14Updated 4 years ago
- Design/Implement stream/batch architecture on NYC taxi data | #DE☆25Updated 4 years ago
- Base Kafka Producer, consumer, flask api and PySpark Structured streaming Job☆11Updated 3 years ago
- Project for real-time anomaly detection using Kafka and python☆58Updated 2 years ago
- Deploying Models to Production with Mlflow and AWS Sagemaker☆22Updated 3 years ago
- Processing TfL data for bike usage with Google Cloud Platform.☆45Updated 3 years ago
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆42Updated last year
- ☆87Updated 2 years ago
- Apache Spark using SQL☆14Updated 3 years ago
- Mastering Big Data Analytics with PySpark, Published by Packt☆161Updated 11 months ago
- plan, design and implement enterprise data infrastructure solutions and create the blueprints for an organization’s data management syste…☆13Updated 2 years ago
- A batch processing data pipeline, using AWS resources (S3, EMR, Redshift, EC2, IAM), provisioned via Terraform, and orchestrated from loc…☆23Updated 3 years ago
- ☆12Updated 4 years ago
- Delta-Lake, ETL, Spark, Airflow☆47Updated 2 years ago
- ☆39Updated 2 years ago
- Writes the CSV file to Postgres, read table and modify it. Write more tables to Postgres with Airflow.☆37Updated last year
- A Series of Notebooks on how to start with Kafka and Python☆153Updated 4 months ago
- Simple ETL pipeline using Python☆26Updated 2 years ago
- A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract d…☆24Updated 3 years ago
- ☆150Updated 7 years ago
- Template for data pipelines, ML workflows, API dev and monitoring☆45Updated last year
- A list of all my posts and personal projects☆73Updated last year
- A real-time streaming ETL pipeline for streaming and performing sentiment analysis on Twitter data using Apache Kafka, Apache Spark and D…☆30Updated 4 years ago
- Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow☆148Updated 5 years ago
- PySpark functions and utilities with examples. Assists ETL process of data modeling☆104Updated 4 years ago
- Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)☆308Updated last year
- Classwork projects and home works done through Udacity data engineering nano degree☆74Updated last year
- Simple stream processing pipeline☆103Updated last year
- Tutorials/use cases of using Prefect in an ML project.☆43Updated 2 years ago