pavank / docker-bigdata-cluster
Infrastructure automation to deploy Hadoop,Hive,Spark,airflow nodes on a docker host
☆20Updated 5 years ago
Related projects: ⓘ
- Hadoop-Hive-Spark cluster + Jupyter on Docker☆53Updated 3 months ago
- Base Docker image with just essentials: Hadoop, Hive and Spark.☆67Updated 3 years ago
- A docker using the airflow with Hadoop ecosystem (hive, spark, and sqoop)☆11Updated 3 years ago
- Dockerizing an Apache Spark Standalone Cluster☆43Updated 2 years ago
- Examples of Spark 3.0☆46Updated 3 years ago
- Example for article Running Spark 3 with standalone Hive Metastore 3.0☆96Updated last year
- Spark Streaming HBase Example☆22Updated 8 years ago
- ETL pipeline using pyspark (Spark - Python)☆106Updated 4 years ago
- A sample implementation of stream writes to an Iceberg table on GCS using Flink and reading it using Trino☆15Updated 2 years ago
- This project describes how to write full ETL data pipeline using spark.☆15Updated last year
- spark on kubernetes☆105Updated last year
- Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi☆106Updated 9 months ago
- ☆23Updated 3 years ago
- Kafka, Spark Streaming, Kudu integration examples☆17Updated 6 years ago
- ☆31Updated 6 years ago
- ☆22Updated 3 years ago
- Docker Big Data Tools: This docker-compose file is configured to run multiple nodes. This is a Hadoop Cluster that contains the necessary…☆27Updated 3 years ago
- Creation of a data lakehouse and an ELT pipeline to enable the efficient analysis and use of data☆37Updated 9 months ago
- Bootstrap a pipeline on the BDE platform☆26Updated 8 years ago
- Spark 3.0.0 Structured Streaming Kafka Avro Demo☆15Updated last year
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆53Updated last year
- Presto Trino with Apache Hive Postgres metastore☆36Updated last week
- Docker image for Apache Hive Metastore☆72Updated last year
- A project with examples of using few commonly used data manipulation/processing/transformation APIs in Apache Spark 2.0.0☆25Updated 3 years ago
- Learning Spark SQL, published by Packt☆40Updated last year
- Spark on Kubernetes using Helm☆34Updated 4 years ago
- Scalable CDC Pattern Implemented using PySpark☆18Updated 5 years ago
- ☆39Updated 5 years ago
- Sample processing code using Spark 2.1+ and Scala☆50Updated 4 years ago
- ☆38Updated this week