Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are initialized. This also allows extending the Spark metrics systems with user-provided monitoring probes.
☆96May 11, 2026Updated 5 months ago
Alternatives and similar repositories for SparkPlugins
Users that are interested in SparkPlugins are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Spark-Dashboard is an open-source monitoring solution for Apache Spark that provides real-time performance dashboards using containers an…☆141Sep 18, 2026Updated last month
- ☆10Jun 29, 2021Updated 5 years ago
- This repository contains the development code for sparkMeasure, an Apache Spark performance analysis and troubleshooting library. It simp…☆832Oct 16, 2026Updated 2 weeks ago
- A library that provides useful extensions to Apache Spark and PySpark.☆241Sep 10, 2026Updated last month
- A library that brings useful functions from various modern database management systems to Apache Spark☆63Sep 4, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- This is a mirror of https://github.com/LucaCanali/sparkMeasure - sparkMeasure is a tool for performance troubleshooting of Apache Spark w…☆16May 21, 2026Updated 5 months ago
- Spark metrics related custom classes and sinks (e.g. Prometheus)☆185Aug 2, 2022Updated 4 years ago
- Spark-Radiant is Apache Spark Performance and Cost Optimizer☆25Dec 31, 2024Updated last year
- Qubole Sparklens tool for performance tuning Apache Spark☆593Jun 26, 2024Updated 2 years ago
- Paper: A Zero-rename committer for object stores☆20Nov 7, 2025Updated 11 months ago
- Includes notes on using Apache Spark, with drill down on Spark for Physics, how to run TPCDS on PySpark, how to create histograms with S…☆464May 19, 2026Updated 5 months ago
- A Spark plugin for CPU and memory profiling☆21Mar 17, 2026Updated 7 months ago
- An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.☆430Jan 14, 2022Updated 4 years ago
- Rocksdb state storage implementation for Structured Streaming.☆17Oct 21, 2020Updated 6 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.☆255Feb 21, 2023Updated 3 years ago
- Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.☆1,613Updated this week
- Apache DataFusion Comet Spark Accelerator☆1,310Updated this week
- A Python Library to support running data quality rules while the spark job is running⚡☆203Oct 7, 2026Updated 3 weeks ago
- REST job server for Apache Spark☆43Aug 8, 2026Updated 2 months ago
- Spark SQL index for Parquet tables☆134May 6, 2021Updated 5 years ago
- Essential Spark extensions and helper methods ✨😲☆766Oct 1, 2026Updated last month
- The Internals of Delta Lake☆186Jun 18, 2026Updated 4 months ago
- ACID Data Source for Apache Spark based on Hive ACID☆97Jul 7, 2021Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Expressive types for Spark.☆898Updated this week
- A framework for writing performant user-defined functions (UDFs) that are portable across a variety of engines including Apache Spark, Ap…☆306Jul 30, 2026Updated 3 months ago
- This project provides a reverse proxy for Spark UI on Kubernetes☆16Oct 12, 2023Updated 3 years ago
- Accelerate Spark Processes with GraalVM EE☆10Apr 13, 2021Updated 5 years ago
- A tool to validate data, built around Apache Spark.☆101Aug 10, 2026Updated 2 months ago
- Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.☆1,840Updated this week
- Typesafe wrapper for Apache Spark DataFrame API☆143Jan 24, 2026Updated 9 months ago
- A simplified, lightweight ETL Framework based on Apache Spark☆589Sep 1, 2026Updated 2 months ago
- JupyterLab extension that enables monitoring launched Apache Spark jobs from within a notebook☆92Dec 27, 2022Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- DataTunnel 是一个基于spark引擎的超高性能的分布式数据集成软件，支持海量数据的同步。基于spark extensions 扩展的DSL语法，结合的Spark SQL，更加便捷融入数仓 ETLT 过程中，简单易用。☆36Oct 19, 2026Updated 2 weeks ago
- Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)☆455Apr 2, 2026Updated 7 months ago
- spark structured streaming via HTTP communication☆19Jul 7, 2022Updated 4 years ago
- Sample processing code using Spark 2.1+ and Scala☆51Jun 28, 2020Updated 6 years ago
- Easy CPU Profiling for Apache Spark applications☆49Dec 17, 2025Updated 10 months ago
- type-class based data cleansing library for Apache Spark SQL☆78Jun 23, 2019Updated 7 years ago
- Shed light on your data layout in order to monitor the health of your Lakehouse tables and identify when data maintenance operations shou…☆10Jul 31, 2023Updated 3 years ago