The goal of this project is to build a docker cluster that gives access to Hadoop, HDFS, Hive, PySpark, Sqoop, Airflow, Kafka, Flume, Postgres, Cassandra, Hue, Zeppelin, Kadmin, Kafka Control Center and pgAdmin. This cluster is solely intended for usage in a development environment. Do not use it to run any production workloads.
☆80Feb 27, 2023Updated 3 years ago
Alternatives and similar repositories for Big-Data-Cluster
Users that are interested in Big-Data-Cluster are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Hadoop-Hive-Spark cluster + Jupyter on Docker☆84Jan 2, 2025Updated last year
- This project demonstrates real-time data streaming and processing architecture using Kafka, Spark Streaming, and Debezium for capturing C…☆15Oct 24, 2024Updated last year
- ☆15May 1, 2024Updated 2 years ago
- I implemented various ETL processes like loading the data using sqoop from mysql to hdfs, transform the data using Spark and Scala, perfo…☆10Oct 20, 2017Updated 8 years ago
- This project aims to move the data from a Relational database system (RDBMS) to a Hadoop file system (HDFS)☆11Apr 29, 2022Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆163Dec 27, 2024Updated last year
- ClickStack - the open-source observability stack for OpenTelemetry at scale☆116Feb 26, 2026Updated 4 months ago
- ☆23Feb 5, 2024Updated 2 years ago
- ☆17Jul 10, 2022Updated 3 years ago
- Base hadoop/spark/bigdata image with advanced config loading scripts.☆11Nov 3, 2020Updated 5 years ago
- Delta-Lake, ETL, Spark, Airflow☆50Oct 9, 2022Updated 3 years ago
- This repo is for generating data from existing dataset to a file or producing dataset rows as message to kafka in a streaming manner.☆22May 16, 2026Updated last month
- Here I will be exploring various tools and methods that are used in data engineering process with Python.☆21Jan 4, 2021Updated 5 years ago
- Bu repo 3-5 gün süreyle düzenlenen Python ile Makine Öğrenmesi Eğitimleri için oluşturulmuştur.☆20Oct 9, 2020Updated 5 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- an example of how to handle the gowalla check in dataset☆14May 25, 2017Updated 9 years ago
- Data engineering mentorship program☆207Feb 21, 2026Updated 4 months ago
- Extract, transform, and load data for analytic processing using AWS Glue☆17May 2, 2021Updated 5 years ago
- Series follows learning from Apache Spark (PySpark) with quick tips and workaround for daily problems in hand☆55Sep 30, 2023Updated 2 years ago
- Project - Data Processing and Analysis in Python Course☆39Oct 10, 2018Updated 7 years ago
- Dockerizing an Apache Spark Standalone Cluster☆42Jun 29, 2022Updated 4 years ago
- Data pipeline for extracting, transforming, and visualising Covid-19 data☆14Apr 23, 2023Updated 3 years ago
- ☆24Dec 31, 2024Updated last year
- ☆12Jul 27, 2021Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆10Jul 24, 2024Updated last year
- Small data engineering tutorial☆10Oct 24, 2018Updated 7 years ago
- Tutorial and examples for using Apache Spark☆18Jul 21, 2017Updated 8 years ago
- Now updated prior to the version on CRAN.☆15Jan 9, 2024Updated 2 years ago
- ☆17Apr 1, 2025Updated last year
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆45Jan 4, 2024Updated 2 years ago
- A shell script to automate the operations of sqoop☆11Mar 29, 2021Updated 5 years ago
- It is a assemble to include all Practice Projects about Big Data Topic, includes Hadoop, Spark, Spark Streaming and Kafka☆11Mar 7, 2019Updated 7 years ago
- Lightweight .NET Core library for generating SQL statements from plain objects with advanced features and mapping support☆14Feb 2, 2018Updated 8 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Node-RED Flow (and web page example) for the LLaMA AI model☆11Jul 27, 2023Updated 2 years ago
- ☆97Mar 8, 2024Updated 2 years ago
- TTS utility☆12Aug 2, 2020Updated 5 years ago
- Generate and Compare Debezium CDC (Chance Data Capture) Avro Schema, directly from your Database.☆28Jul 1, 2026Updated last week
- Extract transform load CLI tool for extracting small and middle data volume from sources (databases, csv files, xls files, gspreadsheets)…☆11Jun 30, 2026Updated last week
- A partially implemented ODBC driver for the Trino distributed SQL engine☆20Jun 2, 2026Updated last month
- audio, NLP, ML with huggingface, nvidia/nemo, speechbrain☆11Sep 4, 2023Updated 2 years ago