The goal of this project is to build a docker cluster that gives access to Hadoop, HDFS, Hive, PySpark, Sqoop, Airflow, Kafka, Flume, Postgres, Cassandra, Hue, Zeppelin, Kadmin, Kafka Control Center and pgAdmin. This cluster is solely intended for usage in a development environment. Do not use it to run any production workloads.
☆80Feb 27, 2023Updated 3 years ago
Alternatives and similar repositories for Big-Data-Cluster
Users that are interested in Big-Data-Cluster are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Hadoop-Hive-Spark cluster + Jupyter on Docker☆84Jan 2, 2025Updated last year
- Repository for building docker image, with open-source applications☆26Apr 23, 2024Updated 2 years ago
- Run Hadoop Cluster within Docker Containers.☆16Mar 6, 2025Updated last year
- I implemented various ETL processes like loading the data using sqoop from mysql to hdfs, transform the data using Spark and Scala, perfo…☆10Oct 20, 2017Updated 8 years ago
- Spark all the ETL Pipelines☆37Aug 2, 2023Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- This project demonstrates real-time data streaming and processing architecture using Kafka, Spark Streaming, and Debezium for capturing C…☆16Oct 24, 2024Updated last year
- ☆23Feb 5, 2024Updated 2 years ago
- Final Year Project: EPOS web application implementing an electronic point of sale interface, sales analytics, sales weekly/monthly/yearl…☆16Dec 9, 2021Updated 4 years ago
- Delta-Lake, ETL, Spark, Airflow☆50Oct 9, 2022Updated 3 years ago
- Bu repo 3-5 gün süreyle düzenlenen Python ile Makine Öğrenmesi Eğitimleri için oluşturulmuştur.☆20Oct 9, 2020Updated 5 years ago
- Airflow Examples: code samples for Medium articles☆14Jan 10, 2021Updated 5 years ago
- A basic Text2SQL App, powered by Langchain and OpenAI.☆12Jun 27, 2024Updated 2 years ago
- Comet Busters! 1994 remake using SDL☆12Nov 21, 2022Updated 3 years ago
- Repositorio utilizado para el Curso de Hadoop en Platzi☆11Dec 9, 2020Updated 5 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Curso de Swarm☆12Nov 16, 2018Updated 7 years ago
- PoS crypto coin over ipfs distributed storage network (with new consensus protocol 🙌)☆15Apr 10, 2024Updated 2 years ago
- A shell script to automate the operations of sqoop☆11Mar 29, 2021Updated 5 years ago
- Scalable OLAP system for credit card transaction analysis, leveraging AWS S3, Databricks, and dbt. Features end-to-end batch processing p…☆26Oct 24, 2024Updated last year
- Marshmallow serializer integration with pyspark☆12Dec 29, 2023Updated 2 years ago
- Not everyone can code, but everyone can learn. This Project is an AI powered DSA/Competitive Programming Helper with an inbuilt editor to…☆14Jun 3, 2025Updated last year
- ☆21Mar 11, 2025Updated last year
- Extract, transform, and load data for analytic processing using AWS Glue☆17May 2, 2021Updated 5 years ago
- ☆12Jul 22, 2025Updated last year
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- The repository for my talk titled the same☆15Nov 20, 2019Updated 6 years ago
- Cloud Functions streaming insert to BigQuery (with Cloud Pub/Sub trigger). In this example, the function will make a REST API call to get…☆29Aug 28, 2023Updated 2 years ago
- ☆11Sep 28, 2022Updated 3 years ago
- A Python PySpark Projet with Poetry☆31May 2, 2026Updated 2 months ago
- Spark implementation of Slowly Changing Dimension type 2☆11Jan 8, 2019Updated 7 years ago
- TTS utility☆12Aug 2, 2020Updated 5 years ago
- A buildroot based Raspberry Pi emulator platform☆22Feb 7, 2015Updated 11 years ago
- Small data engineering tutorial☆10Oct 24, 2018Updated 7 years ago
- Dockerizing an Apache Spark Standalone Cluster☆42Jun 29, 2022Updated 4 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Docker powered container for using Nginx as reverse-proxy in combination with an OpenVPN Client.☆11Jan 1, 2020Updated 6 years ago
- An open source enterprise data warehousing and analysis platform.☆23Nov 8, 2021Updated 4 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆45Jan 4, 2024Updated 2 years ago
- La regresión linear es un algoritmo clásico de machine learning que es muy fácil de implementar. Sin embargo, es muy poderoso y tiene muc…☆14Jan 24, 2021Updated 5 years ago
- Big Data Inventory Management on AWS (Demand Forecasting, Machine Learning, Dashboarding) : Presented at Carlson School of Management dur…☆11Apr 15, 2020Updated 6 years ago
- Todo App with nodeJs, Express, Mongoose and MongoDB☆20Jun 27, 2026Updated last month
- A flake8 plugin that detects of usage withColumn in a loop or inside reduce☆28Jun 20, 2025Updated last year