sdesilva26 / docker-sparkLinks
Tutorial for setting up a Spark cluster running inside of Docker containers located on different machines
☆135Updated 3 years ago
Alternatives and similar repositories for docker-spark
Users that are interested in docker-spark are comparing it to the libraries listed below
Sorting:
- Learn Apache Spark in Scala, Python (PySpark) and R (SparkR) by building your own cluster with a JupyterLab interface on Docker.☆507Updated 3 months ago
- Docker with Airflow and Spark standalone cluster☆262Updated 2 years ago
- Delta Lake examples☆238Updated last year
- A workspace to experiment with Apache Spark, Livy, and Airflow in a Docker environment.☆38Updated 4 years ago
- A simple spark standalone cluster for your testing environment purposses☆569Updated last year
- ☆270Updated last year
- Multi-container environment with Hadoop, Spark and Hive☆232Updated 9 months ago
- Delta Lake helper methods in PySpark☆327Updated 3 weeks ago
- A Python PySpark Projet with Poetry☆24Updated 6 months ago
- O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian☆228Updated 2 years ago
- Spark style guide☆271Updated last year
- Streaming Synthetic Sales Data Generator: Streaming sales data generator for Apache Kafka, written in Python☆44Updated 3 years ago
- spark on kubernetes☆104Updated 2 years ago
- Delta Lake Documentation☆53Updated last year
- Apache Airflow in Docker Compose (for both versions 1.10.* and 2.*)☆185Updated 2 years ago
- ☆60Updated last year
- PySpark test helper methods with beautiful error messages☆752Updated 3 weeks ago
- Example of how to leverage Apache Spark distributed capabilities to call REST-API using a UDF☆52Updated 3 years ago
- Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validatio…☆56Updated 2 years ago
- Simple stream processing pipeline☆110Updated last year
- Companion repository for the book 'Delta Lake Up and Running'☆48Updated 10 months ago
- Databricks - Apache Spark™ - 2X Certified Developer☆266Updated 5 years ago
- Apache Spark 3 - Structured Streaming Course Material☆126Updated 2 years ago
- Delta-Lake, ETL, Spark, Airflow☆48Updated 3 years ago
- ☆93Updated last year
- Data Engineering with Spark and Delta Lake☆106Updated 3 years ago
- Simple repo to demonstrate how to submit a spark job to EMR from Airflow☆34Updated 5 years ago
- pyspark methods to enhance developer productivity 📣 👯 🎉☆682Updated 11 months ago
- A Python Library to support running data quality rules while the spark job is running⚡☆197Updated this week
- Execution of DBT models using Apache Airflow through Docker Compose☆126Updated 3 years ago