Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are initialized. This also allows extending the Spark metrics systems with user-provided monitoring probes.
☆96May 11, 2026Updated 4 months ago
Alternatives and similar repositories for SparkPlugins
Users that are interested in SparkPlugins are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Spark-Dashboard is an open-source monitoring solution for Apache Spark that provides real-time performance dashboards using containers an…☆140Updated this week
- ☆10Jun 29, 2021Updated 5 years ago
- This repository contains the development code for sparkMeasure, an Apache Spark performance analysis and troubleshooting library. It simp…☆829May 19, 2026Updated 4 months ago
- A library that brings useful functions from various modern database management systems to Apache Spark☆63Sep 4, 2023Updated 3 years ago
- A library that provides useful extensions to Apache Spark and PySpark.☆240Sep 10, 2026Updated last week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Spark-Radiant is Apache Spark Performance and Cost Optimizer☆25Dec 31, 2024Updated last year
- Spark metrics related custom classes and sinks (e.g. Prometheus)☆186Aug 2, 2022Updated 4 years ago
- This is a mirror of https://github.com/LucaCanali/sparkMeasure - sparkMeasure is a tool for performance troubleshooting of Apache Spark w…☆16May 21, 2026Updated 4 months ago
- Paper: A Zero-rename committer for object stores☆20Nov 7, 2025Updated 10 months ago
- Qubole Sparklens tool for performance tuning Apache Spark☆593Jun 26, 2024Updated 2 years ago
- On the fly, translation of Spark programs to run natively on your Oracle DB. Your Spark programs require no changes.☆36Apr 15, 2025Updated last year
- Includes notes on using Apache Spark, with drill down on Spark for Physics, how to run TPCDS on PySpark, how to create histograms with S…☆465May 19, 2026Updated 4 months ago
- Rocksdb state storage implementation for Structured Streaming.☆17Oct 21, 2020Updated 5 years ago
- Spark SQL index for Parquet tables☆134May 6, 2021Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- NVIDIA cuDF for Apache Spark plugin - accelerate Apache Spark with GPUs☆1,005Updated this week
- REST job server for Apache Spark☆44Aug 8, 2026Updated last month
- Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.☆1,600Updated this week
- An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.☆430Jan 14, 2022Updated 4 years ago
- The Internals of Delta Lake☆186Jun 18, 2026Updated 3 months ago
- Apache DataFusion Comet Spark Accelerator☆1,283Updated this week
- A tool to validate data, built around Apache Spark.☆101Aug 10, 2026Updated last month
- Essential Spark extensions and helper methods ✨😲☆767Jun 22, 2026Updated 3 months ago
- Typesafe wrapper for Apache Spark DataFrame API☆142Jan 24, 2026Updated 7 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Azure Synapse Spark Metrics provides easy metrics monitoring functions for Synapse services, especially, Apache Spark pool instances, by …☆24May 23, 2023Updated 3 years ago
- Expressive types for Spark.☆895Sep 14, 2026Updated last week
- Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.☆255Feb 21, 2023Updated 3 years ago
- Sample processing code using Spark 2.1+ and Scala☆51Jun 28, 2020Updated 6 years ago
- Using log4j insert log info into ElasticSearch☆26Oct 31, 2016Updated 9 years ago
- ACID Data Source for Apache Spark based on Hive ACID☆97Jul 7, 2021Updated 5 years ago
- Mirror of Apache DataFu☆124Jul 9, 2026Updated 2 months ago
- Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)☆457Apr 2, 2026Updated 5 months ago
- type-class based data cleansing library for Apache Spark SQL☆78Jun 23, 2019Updated 7 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Shed light on your data layout in order to monitor the health of your Lakehouse tables and identify when data maintenance operations shou…☆10Jul 31, 2023Updated 3 years ago
- A Python package to submit and manage Apache Spark applications on Kubernetes.☆47Feb 27, 2026Updated 6 months ago
- Apache Auron(Incubating) is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.☆1,809Updated this week
- The Internals of Spark SQL☆491Jan 25, 2026Updated 7 months ago
- spark structured streaming via HTTP communication☆18Jul 7, 2022Updated 4 years ago
- Go Client for Hive Metastore☆14Dec 18, 2022Updated 3 years ago
- ☆17Mar 19, 2024Updated 2 years ago