dbusteed / kafka-spark-streaming-example
☆37Updated 5 years ago
Alternatives and similar repositories for kafka-spark-streaming-example:
Users that are interested in kafka-spark-streaming-example are comparing it to the libraries listed below
- A real-time streaming ETL pipeline for streaming and performing sentiment analysis on Twitter data using Apache Kafka, Apache Spark and D…☆29Updated 4 years ago
- Apche Spark Structured Streaming with Kafka using Python(PySpark)☆41Updated 5 years ago
- Apache Spark 3 - Structured Streaming Course Material☆121Updated last year
- Design/Implement stream/batch architecture on NYC taxi data | #DE☆26Updated 3 years ago
- ☆33Updated 4 years ago
- PySpark Cheatsheet☆35Updated 2 years ago
- Project for real-time anomaly detection using Kafka and python☆57Updated 2 years ago
- Dockerizing an Apache Spark Standalone Cluster☆43Updated 2 years ago
- ☆40Updated 6 months ago
- A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.☆39Updated 3 years ago
- Testing Spark Structured Streaming anf Kafka with real data from traffic sensors☆16Updated 2 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- Classwork projects and home works done through Udacity data engineering nano degree☆74Updated last year
- Creation of a data lakehouse and an ELT pipeline to enable the efficient analysis and use of data☆41Updated last year
- PySpark functions and utilities with examples. Assists ETL process of data modeling☆99Updated 4 years ago
- RedditR for Content Engagement and Recommendation☆21Updated 7 years ago
- Data engineering interviews Q&A for data community by data community☆62Updated 4 years ago
- Simple stream processing pipeline☆94Updated 7 months ago
- ☆86Updated 2 years ago
- Code examples on Apache Spark using python☆106Updated 2 years ago
- Kafka variant of the MLOps Level 1 stack☆22Updated 2 years ago
- Series follows learning from Apache Spark (PySpark) with quick tips and workaround for daily problems in hand☆45Updated last year
- PySpark Tutorial for Beginners on Google Colab: Hands-On Guide☆16Updated 4 years ago
- This project aims to build a streaming application to perform real-time analytics of Covid-19 related tweets and deploy an ML model for r…☆12Updated 3 years ago
- ☆34Updated last year
- Final Project for IoT: Big Data Processing and Analytics class. Analyzing U.S nationwide temperature from IoT sensors in real-time☆69Updated 8 years ago
- My solutions for the Udacity Data Engineering Nanodegree☆33Updated 5 years ago
- Docker with Airflow + Postgres + Spark cluster + JDK (spark-submit support) + Jupyter Notebooks☆21Updated 2 years ago
- pyspark framework☆25Updated 2 years ago
- Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and MLFlow'☆118Updated last year