The goal of this project is to build a docker cluster that gives access to Hadoop, HDFS, Hive, PySpark, Sqoop, Airflow, Kafka, Flume, Postgres, Cassandra, Hue, Zeppelin, Kadmin, Kafka Control Center and pgAdmin. This cluster is solely intended for usage in a development environment. Do not use it to run any production workloads.
☆81Feb 27, 2023Updated 3 years ago
Alternatives and similar repositories for Big-Data-Cluster
Users that are interested in Big-Data-Cluster are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Hadoop-Hive-Spark cluster + Jupyter on Docker☆84Jan 2, 2025Updated last year
- Repository for building docker image, with open-source applications☆26Apr 23, 2024Updated 2 years ago
- This project demonstrates real-time data streaming and processing architecture using Kafka, Spark Streaming, and Debezium for capturing C…☆19Oct 24, 2024Updated last year
- This project aims to move the data from a Relational database system (RDBMS) to a Hadoop file system (HDFS)☆11Apr 29, 2022Updated 4 years ago
- This repo is for generating data from existing dataset to a file or producing dataset rows as message to kafka in a streaming manner.☆22May 16, 2026Updated 3 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆23Feb 5, 2024Updated 2 years ago
- ☆174Dec 27, 2024Updated last year
- ☆15May 1, 2024Updated 2 years ago
- Airflow Examples: code samples for Medium articles☆14Jan 10, 2021Updated 5 years ago
- Spark application to consume kafka events generated by a python producer.☆12Aug 7, 2021Updated 5 years ago
- A basic Text2SQL App, powered by Langchain and OpenAI.☆12Jun 27, 2024Updated 2 years ago
- This Repo contains Jupyter Notebooks to recap on RDD, DataFrame, Spark Streaming and ML operations using Pyspark☆11Nov 3, 2024Updated last year
- Scalable OLAP system for credit card transaction analysis, leveraging AWS S3, Databricks, and dbt. Features end-to-end batch processing p…☆26Oct 24, 2024Updated last year
- ☆20Mar 11, 2025Updated last year
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Extract, transform, and load data for analytic processing using AWS Glue☆17May 2, 2021Updated 5 years ago
- This is end to end data engineering project using dbt , snowflake and apache airflow☆23Apr 28, 2025Updated last year
- Series follows learning from Apache Spark (PySpark) with quick tips and workaround for daily problems in hand☆55Sep 30, 2023Updated 2 years ago
- R code to perform the Lee Strazicich unit root test☆10Aug 8, 2017Updated 9 years ago
- The repository for my talk titled the same☆15Nov 20, 2019Updated 6 years ago
- R package for Markov regime-switching models☆12Jan 23, 2018Updated 8 years ago
- ☆27Dec 31, 2024Updated last year
- This repo gives an introduction to setting up streaming analytics using open source technologies☆25Mar 2, 2023Updated 3 years ago
- Spark implementation of Slowly Changing Dimension type 2☆10Jan 8, 2019Updated 7 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Project - Data Processing and Analysis in Python Course☆39Oct 10, 2018Updated 7 years ago
- TTS utility☆12Aug 2, 2020Updated 6 years ago
- ☆10Jul 24, 2024Updated 2 years ago
- A buildroot based Raspberry Pi emulator platform☆22Feb 7, 2015Updated 11 years ago
- Parameter Importance according to OpenML☆14Feb 23, 2022Updated 4 years ago
- audio, NLP, ML with huggingface, nvidia/nemo, speechbrain☆11Sep 4, 2023Updated 3 years ago
- Dockerizing an Apache Spark Standalone Cluster☆42Jun 29, 2022Updated 4 years ago
- Now updated prior to the version on CRAN.☆15Jan 9, 2024Updated 2 years ago
- Solar Resource Assessment in Python☆12Jul 27, 2026Updated last month
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- An open source enterprise data warehousing and analysis platform.☆23Nov 8, 2021Updated 4 years ago
- This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenA…☆45Jan 4, 2024Updated 2 years ago
- La regresión linear es un algoritmo clásico de machine learning que es muy fácil de implementar. Sin embargo, es muy poderoso y tiene muc…☆14Jan 24, 2021Updated 5 years ago
- Big Data Inventory Management on AWS (Demand Forecasting, Machine Learning, Dashboarding) : Presented at Carlson School of Management dur…☆11Apr 15, 2020Updated 6 years ago
- This repository contains an end-to-end data engineering project using Apache Flink, focused on performing sales analytics. The project de…☆12Nov 18, 2023Updated 2 years ago
- All-in-one Speech Transcription☆11Jun 5, 2026Updated 3 months ago
- Data engineering mentorship program☆221Feb 21, 2026Updated 6 months ago