Includes notes on using Apache Spark, with drill down on Spark for Physics, how to run TPCDS on PySpark, how to create histograms with Spark. Also tools for stress testing, measuring CPUs' performance, and I/O latency heat maps. Jupyter notebooks examples for using various DB systems.
☆464May 19, 2026Updated 5 months ago
Alternatives and similar repositories for Miscellaneous
Users that are interested in Miscellaneous are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This repository contains the development code for sparkMeasure, an Apache Spark performance analysis and troubleshooting library. It simp…☆832Oct 16, 2026Updated 2 weeks ago
- Spark-Dashboard is an open-source monitoring solution for Apache Spark that provides real-time performance dashboards using containers an…☆141Sep 18, 2026Updated last month
- Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are in…☆96May 11, 2026Updated 5 months ago
- Spark metrics related custom classes and sinks (e.g. Prometheus)☆185Aug 2, 2022Updated 4 years ago
- A library that provides useful extensions to Apache Spark and PySpark.☆241Sep 10, 2026Updated last month
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- This is a mirror of https://github.com/LucaCanali/sparkMeasure - sparkMeasure is a tool for performance troubleshooting of Apache Spark w…☆16May 21, 2026Updated 5 months ago
- Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.☆1,613Updated this week
- Hadoop Profiler, or hprofiler, is a tool which is able to analyze on- and off-CPU workloads on distributed computing environments.☆24Jul 7, 2016Updated 10 years ago
- Custom Alerts for Ambari server☆12Jul 27, 2015Updated 11 years ago
- The Internals of Delta Lake☆186Jun 18, 2026Updated 4 months ago
- Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.☆255Feb 21, 2023Updated 3 years ago
- Splash, a flexible Spark shuffle manager that supports user-defined storage backends for shuffle data storage and exchange☆130Dec 19, 2024Updated last year
- Intel® Performance Counter Monitor (Intel® PCM)☆3,350Oct 15, 2026Updated 2 weeks ago
- The Internals of Spark SQL☆491Oct 12, 2026Updated 2 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Provides a set of benchmarks that can be used to measure the memory bandwidth performance of CPU's☆95Apr 8, 2024Updated 2 years ago
- Bulletproof Apache Spark jobs with fast root cause analysis of failures.☆73Mar 14, 2021Updated 5 years ago
- Scripts for generating Grafana dashboards for monitoring Spark jobs☆238Mar 26, 2015Updated 11 years ago
- Tool for gathering blocks and replicas meta data from HDFS. It also builds a heat map showing how replicas are distributed along disks an…☆55May 9, 2017Updated 9 years ago
- An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.☆430Jan 14, 2022Updated 4 years ago
- Jupyter magics and kernels for working with remote Spark clusters☆1,369Sep 9, 2025Updated last year
- Monitor Apache Spark from Jupyter Notebook☆172May 16, 2022Updated 4 years ago
- The Internals of Apache Spark☆1,552Oct 10, 2026Updated 2 weeks ago
- A Spark Reliability Testing Suite☆13Jan 10, 2017Updated 9 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A re-implementation of Hadoop DistCP in Apache Spark☆47Dec 20, 2023Updated 2 years ago
- Helpers & syntactic sugar for PySpark.☆62Sep 11, 2026Updated last month
- JVM Profiler Sending Metrics to Kafka, Console Output or Custom Reporter☆1,801May 21, 2026Updated 5 months ago
- Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark☆1,370Aug 22, 2023Updated 3 years ago
- ☆313Nov 26, 2018Updated 7 years ago
- Collection of oracle scripts☆25Mar 3, 2020Updated 6 years ago
- Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.☆1,078Updated this week
- Takes AWR reports as input and produces 1-page summary of key instrumentation data to identify and help reason about inconsistencies☆16Feb 22, 2019Updated 7 years ago
- A Spark metrics sink that pushes to InfluxDb☆51Jan 14, 2021Updated 5 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A Spark UI and Spark History Server alternative with CPU and Memory metrics! Delight is free, cross-platform, and open-source.☆344May 31, 2024Updated 2 years ago
- NVIDIA cuDF for Apache Spark plugin - accelerate Apache Spark with GPUs☆1,012Updated this week
- User space software for Intel(R) Resource Director Technology☆755Updated this week
- A tool to get better debug info on spark's memory usage☆42Aug 21, 2019Updated 7 years ago
- Allows reading ROOT TTrees into Apache Spark as DataFrames☆12Jun 14, 2023Updated 3 years ago
- ☆11Feb 14, 2020Updated 6 years ago
- REST job server for Apache Spark☆2,833Mar 3, 2026Updated 7 months ago