rogaha / data-processing-pipeline
Real-Time Data Processing Pipeline & Visualization with Docker, Spark, Kafka and Cassandra
☆83Updated 7 years ago
Related projects ⓘ
Alternatives and complementary repositories for data-processing-pipeline
- Docker container for Kafka - Spark Streaming - Cassandra☆97Updated 5 years ago
- Real-world Spark pipelines examples☆83Updated 6 years ago
- How to manage Slowly Changing Dimensions with Apache Hive☆55Updated 5 years ago
- An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR☆173Updated last year
- This project describes how to write full ETL data pipeline using spark.☆15Updated 2 years ago
- Spark structured streaming with Kafka data source and writing to Cassandra☆64Updated 4 years ago
- A stack overflow for Apache Spark☆72Updated 7 years ago
- An example Apache Beam project.☆111Updated 7 years ago
- Code examples and docker environment for Spark☆26Updated 8 years ago
- Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University☆154Updated last week
- real time log event processing using spark, kafka & cassandra☆13Updated 9 years ago
- These are some code examples☆55Updated 4 years ago
- Simple examle for Spark Streaming over Kafka topic☆107Updated 4 years ago
- ☆47Updated 4 years ago
- A plugin to Apache Airflow to allow you to run Spark Submit Commands as an Operator☆75Updated 5 years ago
- Apache Spark ETL Utilities☆40Updated 3 weeks ago
- Real-time Machine Learning with Apache Spark on Twitter Public Stream☆68Updated 7 years ago
- A curated list of all the awesome examples, articles, tutorials and videos for Apache Airflow.☆96Updated 3 years ago
- Apache Spark (Scala, PySpark, SparkR) Code, Tricks, and References☆70Updated 5 years ago
- A light Kafka to HDFS/S3 ETL library based on Apache Spark☆41Updated 7 years ago
- Event data simulator. Generates a stream of pseudo-random events from a set of users, designed to simulate web traffic.☆27Updated 7 years ago
- A Spark Streaming job reading events from Amazon Kinesis and writing event counts to DynamoDB☆94Updated 4 years ago
- How to build an awesome data engineering team☆99Updated 5 years ago
- Repository used for Spark Trainings☆53Updated last year
- A proof of concept using Divolte, Kafka, Druid and Superset☆61Updated 4 years ago
- Spark Structured Streaming / Kafka / Cassandra / Elastic☆184Updated last year
- PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2☆83Updated 4 years ago
- ☆28Updated 4 years ago
- A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python,…☆209Updated 5 years ago