jaumpedro214 / traffic-flow-spark-kafkaLinks
Testing Spark Structured Streaming anf Kafka with real data from traffic sensors
☆17Updated 3 years ago
Alternatives and similar repositories for traffic-flow-spark-kafka
Users that are interested in traffic-flow-spark-kafka are comparing it to the libraries listed below
Sorting:
- ☆37Updated 6 years ago
- Stream/batch system with Hadoop, Spark on NYC taxi data | #DE☆26Updated 3 months ago
- Base Kafka Producer, consumer, flask api and PySpark Structured streaming Job☆11Updated 4 years ago
- RedditR for Content Engagement and Recommendation☆18Updated 8 years ago
- ☆88Updated 3 years ago
- Spark, Airflow, Kafka☆24Updated 2 years ago
- ☆45Updated last year
- Processing TfL data for bike usage with Google Cloud Platform.☆46Updated 3 years ago
- Mastering Big Data Analytics with PySpark, Published by Packt☆164Updated last year
- Final Project for IoT: Big Data Processing and Analytics class. Analyzing U.S nationwide temperature from IoT sensors in real-time☆71Updated 9 years ago
- This project aims to build a streaming application to perform real-time analytics of Covid-19 related tweets and deploy an ML model for r…☆14Updated 4 years ago
- Project for real-time anomaly detection using Kafka and python☆58Updated 3 years ago
- Series follows learning from Apache Spark (PySpark) with quick tips and workaround for daily problems in hand☆56Updated 2 years ago
- PySpark functions and utilities with examples. Assists ETL process of data modeling☆104Updated 5 years ago
- Simple ETL pipeline using Python☆29Updated 2 years ago
- Apache Spark 3 - Structured Streaming Course Material☆125Updated 2 years ago
- My solutions for the Udacity Data Engineering Nanodegree☆34Updated 6 years ago
- Apache Spark using SQL☆14Updated 4 years ago
- Source code for the Apache Kafka in Python video series☆19Updated 3 years ago
- Course Material Data Engineering on AWS Course☆31Updated last year
- A batch processing data pipeline, using AWS resources (S3, EMR, Redshift, EC2, IAM), provisioned via Terraform, and orchestrated from loc…☆23Updated 3 years ago
- Developed an ETL pipeline for a Data Lake that extracts data from S3, processes the data using Spark, and loads the data back into S3 as …☆16Updated 6 years ago
- Data Streaming Nanodegree (from Udacity) exercises, projects and their solutions☆17Updated 2 years ago
- This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging…☆96Updated 6 years ago
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.☆43Updated 2 years ago
- Deploying Models to Production with Mlflow and AWS Sagemaker☆24Updated 4 years ago
- A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract d…☆24Updated 4 years ago
- ☆12Updated 4 years ago
- code snippet for analytics sessions☆34Updated 3 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆55Updated 2 years ago