apssouza22 / big-data-pipeline-lambda-arch
A full big data pipeline (Lambda Architecture) with Spark, Kafka, HDFS and Cassandra.
☆177Updated 2 years ago
Alternatives and similar repositories for big-data-pipeline-lambda-arch:
Users that are interested in big-data-pipeline-lambda-arch are comparing it to the libraries listed below
- This is the central repository for all materials related to Kafka Streams : Real-time Stream Processing! Book by Prashant Pandey.☆165Updated 4 years ago
- Real-world Spark pipelines examples☆83Updated 7 years ago
- How to build an awesome data engineering team☆100Updated 5 years ago
- Apche Spark Structured Streaming with Kafka using Python(PySpark)☆40Updated 5 years ago
- Building Big Data Pipelines with Apache Beam, published by Packt☆86Updated 2 years ago
- This repository contains code for Spark Streaming☆21Updated 4 years ago
- Simple stream processing pipeline☆101Updated 10 months ago
- Supporting repository for the blog post at https://medium.com/@stephane.maarek/how-to-use-apache-kafka-to-transform-a-batch-pipeline-into…☆243Updated last year
- ( These solutions tested on 4 node Hortonwork cluster on my laptop. Do not test on your production environment until you test... :)☆21Updated 5 years ago
- Cloudera_Material: Study Material to help people preparing for Cloudera CCA Spark and Hadoop Developer Exam (CCA175). Feel free to collab…☆37Updated 5 years ago
- Educational notes,Hands on problems w/ solutions for hadoop ecosystem☆87Updated 6 years ago
- Build and run Spark Structured Streaming pipelines in Hadoop - project using PySpark.☆13Updated 5 years ago
- ☆14Updated 2 years ago
- Spark on Kubernetes using Helm☆34Updated 4 years ago
- ETL pipeline using pyspark (Spark - Python)☆114Updated 5 years ago
- One click deploy docker-compose with Kafka, Spark Streaming, Zeppelin UI and Monitoring (Grafana + Kafka Manager)☆120Updated 3 years ago
- A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python,…☆209Updated 6 years ago
- ☆53Updated 8 months ago
- ☆309Updated 6 years ago
- My Study guide used to pass the CRT020 Spark Certification exam☆33Updated 5 years ago
- 📚 Tech blogs & talks by companies that run Apache Flink in production☆169Updated 3 months ago
- Real Time Big Data / IoT Machine Learning (Model Training and Inference) with HiveMQ (MQTT), TensorFlow IO and Apache Kafka - no addition…☆414Updated 4 years ago
- ☆92Updated 2 years ago
- Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)☆185Updated last year
- Different ways to process data into Cassandra in realtime with technologies such as Kafka, Spark, Akka, Flink☆31Updated 2 years ago
- Code for docker images☆39Updated 2 years ago
- Databricks - Apache Spark™ - 2X Certified Developer☆266Updated 4 years ago
- Maven quick start for building Kafka Connect connectors.☆146Updated 4 years ago
- A simple spark standalone cluster for your testing environment purposses☆571Updated last year
- Examples To Help You Learn Apache Spark☆77Updated 6 years ago