A hybrid Big Data pipeline architecture that combines a real-time streaming layer with a batch layer to process large datasets(Lambda Architecture)
☆191Mar 17, 2026Updated 7 months ago
Alternatives and similar repositories for big-data-pipeline-lambda-arch
Users that are interested in big-data-pipeline-lambda-arch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆10May 1, 2023Updated 3 years ago
- A microservice project leveraging Service Mesh with advanced features from Istio☆43Sep 7, 2025Updated last year
- Developing a Lambda Architecture pipeline using Apache Kafka, Spark Structured Streaming, Redshift, S3, Python☆22Mar 8, 2020Updated 6 years ago
- Kafka in a Container☆14Dec 30, 2021Updated 4 years ago
- ☆24Aug 8, 2021Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- InfluxDB 2 Connector for Kafka☆13Mar 6, 2020Updated 6 years ago
- Pipelines Example Applications☆13Nov 5, 2019Updated 6 years ago
- Built a stream processing data pipeline to get data from disparate systems into a dashboard using Kafka as an intermediary.☆29Aug 14, 2023Updated 3 years ago
- ☆13Jul 16, 2018Updated 8 years ago
- Kafka Connect connector for CDC data from postgres☆11Aug 27, 2017Updated 9 years ago
- Docker example with kafka connect and sink☆12Feb 12, 2018Updated 8 years ago
- This application receives messages from a mqtt broker and sends the messages to a kafka cluster. Topic mapping is configurable.☆30Dec 25, 2025Updated 10 months ago
- Analytics projects using Big Data eco-systems (Hadoop, Spark, Storm)☆17Dec 27, 2021Updated 4 years ago
- A minimal, browseable, read-only, memory-based filesystem in Go.☆12Jul 10, 2019Updated 7 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A benchmarking for python client libraries of Apache Kafka (pycontw 2017)☆17Jun 27, 2017Updated 9 years ago
- ☆15Jan 16, 2018Updated 8 years ago
- Setting up macOS the fast-way. Docs available at https://i3p9.github.io/mac-setup/☆12Oct 15, 2023Updated 3 years ago
- simple ansible role to add a unix user with its ssh key☆10Updated this week
- Docker Big Data Tools: This docker-compose file is configured to run multiple nodes. This is a Hadoop Cluster that contains the necessary…☆31Jul 6, 2021Updated 5 years ago
- B19415 - The Definitive Guide to Data Integration☆10Apr 15, 2024Updated 2 years ago
- A fast Key/Value store implementation using Golang☆12Jun 12, 2017Updated 9 years ago
- In this project I have built etl pipline which scraps the trending repository based on month,week and day LIVE extract other related info…☆12Sep 9, 2023Updated 3 years ago
- 🌟 An end-to-end full-stack data science project, including modelling, MLOps, and data storytelling. ✨☆16Aug 30, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Delta Lake Examples☆11Apr 24, 2020Updated 6 years ago
- ☆21Jun 19, 2019Updated 7 years ago
- http4s middleware for HTTP SPNEGO Authentication☆12Feb 27, 2021Updated 5 years ago
- Spark Structured Streaming data pipeline that processes movie ratings data in real-time.☆14Oct 23, 2026Updated last week
- ☆16Sep 13, 2016Updated 10 years ago
- A Slack ChatOps bot☆13Apr 9, 2025Updated last year
- ☆11Jul 21, 2023Updated 3 years ago
- This is a recipe for docker container based architecture based on airflow, kafka,spark,docker☆19Oct 15, 2024Updated 2 years ago
- Sample Faust project to process tweets in real-time☆13Mar 29, 2021Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Some useful bash scripts to make life easier.☆10Oct 20, 2019Updated 7 years ago
- Dockerised example of monitoring Apache Kafka with Prometheus and Grafana☆146Mar 22, 2019Updated 7 years ago
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Ka…☆369Feb 14, 2025Updated last year
- Architecture Katas☆13Dec 2, 2020Updated 5 years ago
- Go & React Template☆12Oct 13, 2023Updated 3 years ago
- Command Line Tool For Generating Golang Fixtures☆14Dec 29, 2019Updated 6 years ago
- a plugin based grpc framework☆16May 3, 2019Updated 7 years ago