Includes notes on using Apache Spark, with drill down on Spark for Physics, how to run TPCDS on PySpark, how to create histograms with Spark. Also tools for stress testing, measuring CPUs' performance, and I/O latency heat maps. Jupyter notebooks examples for using various DB systems.
☆465May 19, 2026Updated 4 months ago
Alternatives and similar repositories for Miscellaneous
Users that are interested in Miscellaneous are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This repository contains the development code for sparkMeasure, an Apache Spark performance analysis and troubleshooting library. It simp…☆830May 19, 2026Updated 4 months ago
- Spark-Dashboard is an open-source monitoring solution for Apache Spark that provides real-time performance dashboards using containers an…☆140Sep 18, 2026Updated 3 weeks ago
- Qubole Sparklens tool for performance tuning Apache Spark☆593Jun 26, 2024Updated 2 years ago
- Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.☆1,602Updated this week
- The Internals of Delta Lake☆186Jun 18, 2026Updated 3 months ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Spark metrics related custom classes and sinks (e.g. Prometheus)☆185Aug 2, 2022Updated 4 years ago
- Provides a set of benchmarks that can be used to measure the memory bandwidth performance of CPU's☆94Apr 8, 2024Updated 2 years ago
- A library that provides useful extensions to Apache Spark and PySpark.☆241Sep 10, 2026Updated last month
- Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.☆255Feb 21, 2023Updated 3 years ago
- Hadoop Profiler, or hprofiler, is a tool which is able to analyze on- and off-CPU workloads on distributed computing environments.☆24Jul 7, 2016Updated 10 years ago
- The Internals of Spark SQL☆491Jan 25, 2026Updated 8 months ago
- Intel® Performance Counter Monitor (Intel® PCM)☆3,341Updated this week
- Enabling Spark Optimization through Cross-stack Monitoring and Visualization☆47Aug 23, 2017Updated 9 years ago
- Bulletproof Apache Spark jobs with fast root cause analysis of failures.☆73Mar 14, 2021Updated 5 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Helpers & syntactic sugar for PySpark.☆62Sep 11, 2026Updated last month
- The Internals of Apache Spark☆1,551Jul 18, 2026Updated 2 months ago
- Monitor Apache Spark from Jupyter Notebook☆172May 16, 2022Updated 4 years ago
- JVM Profiler Sending Metrics to Kafka, Console Output or Custom Reporter☆1,802May 21, 2026Updated 4 months ago
- This is a mirror of https://github.com/LucaCanali/sparkMeasure - sparkMeasure is a tool for performance troubleshooting of Apache Spark w…☆16May 21, 2026Updated 4 months ago
- Scripts for generating Grafana dashboards for monitoring Spark jobs☆238Mar 26, 2015Updated 11 years ago
- Remote shuffle service for Apache Spark to store shuffle data on remote servers.☆335Sep 29, 2023Updated 3 years ago
- Splash, a flexible Spark shuffle manager that supports user-defined storage backends for shuffle data storage and exchange☆131Dec 19, 2024Updated last year
- User space software for Intel(R) Resource Director Technology☆753Updated this week
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.☆431Jan 14, 2022Updated 4 years ago
- Jupyter magics and kernels for working with remote Spark clusters☆1,367Sep 9, 2025Updated last year
- Queries, Dashboards, and Splunk Knowledge Objects to Monitor Oracle Database Metrics☆14Mar 11, 2021Updated 5 years ago
- ☆14Aug 21, 2021Updated 5 years ago
- A tool to get better debug info on spark's memory usage☆42Aug 21, 2019Updated 7 years ago
- Custom Alerts for Ambari server☆12Jul 27, 2015Updated 11 years ago
- The Internals of Spark Structured Streaming☆419Mar 3, 2026Updated 7 months ago
- Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark☆1,370Aug 22, 2023Updated 3 years ago
- Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.☆1,066Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Demo code showing how to use Java's StructuredTaskScope☆12Sep 29, 2026Updated last week
- NVIDIA cuDF for Apache Spark plugin - accelerate Apache Spark with GPUs☆1,008Updated this week
- ☆313Nov 26, 2018Updated 7 years ago
- spark structured streaming via HTTP communication☆19Jul 7, 2022Updated 4 years ago
- Apache DataFusion Comet Spark Accelerator☆1,288Updated this week
- A dynamic data completeness and accuracy library at enterprise scale for Apache Spark☆30May 13, 2026Updated 4 months ago
- Paper: A Zero-rename committer for object stores☆20Nov 7, 2025Updated 11 months ago