Spark* Shuffle plugin for support shuffling through remote persistent memory over fabrics, which leverages the RDMA network and remote persistent memory (for read) to provide extremely high performance and low latency shuffle solutions for Spark*.
☆14Sep 18, 2023Updated 2 years ago
Alternatives and similar repositories for pmem-shuffle
Users that are interested in pmem-shuffle are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Spark* shuffle plugin for support shuffling data through a remote Hadoop-compatible file system, as opposed to vanilla Spark's local-dis…☆21Mar 15, 2024Updated 2 years ago
- Source code for XPGraph-MICRO22☆11Apr 10, 2023Updated 3 years ago
- Spark Shuffle Optimization with RDMA+AEP☆30May 23, 2023Updated 3 years ago
- Tools for building, packaging, and OAP public cloud integrations such as AWS EMR, Google Dataproc and K8S.☆18Mar 27, 2024Updated 2 years ago
- Spark* plug-in for accelerating Spark* SQL performance by using cache and index at SQL data source layer.☆37Jan 3, 2023Updated 3 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Hadoop InputFormat for http://druid.io/☆10Oct 26, 2016Updated 9 years ago
- ☆12Apr 7, 2025Updated last year
- A Python library and command line utility for manipulating and plotting stellar lightcurves.☆10Jun 14, 2016Updated 10 years ago
- R package implementing the Nearest Neighbor Descent method for approximate nearest neighbors☆18Jul 11, 2026Updated 2 weeks ago
- 基于多线程与epoll的高并发TCP服务器☆11Aug 4, 2018Updated 7 years ago
- ☆12Mar 13, 2024Updated 2 years ago
- Alerting and monitoring tool for Apache Spark☆23May 20, 2022Updated 4 years ago
- High Performance Network Library for RDMA☆28Jan 3, 2023Updated 3 years ago
- HMM-guided metagenomic gene-targeted assembler using iterative de Bruijn graphs☆18Oct 3, 2016Updated 9 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Firestorm is a Remote Shuffle Service, and provides the capability for Apache Spark and Apache Hadoop MapReduce applications to store shu…☆256Apr 7, 2023Updated 3 years ago
- Apache Spark - A unified analytics engine for large-scale data processing☆16Jul 24, 2023Updated 3 years ago
- A Python interface to gb-io, a fast GenBank parser written in Rust.☆24May 21, 2026Updated 2 months ago
- Java implementation of the SCRAM SASL for both server and client plus examples☆17Apr 18, 2021Updated 5 years ago
- benchmark-for-spark☆19May 7, 2025Updated last year
- A S3 Shuffle plugin for Apache Spark to enable elastic scaling for generic Spark workloads.☆52Sep 17, 2025Updated 10 months ago
- Passive Disaggregated Persistent Memory at USENIX ATC 2020.☆52Nov 2, 2020Updated 5 years ago
- ☆18Dec 30, 2025Updated 6 months ago
- Text Preprocessing in Python☆19Jan 15, 2017Updated 9 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Remote Persistent Memory Access Library☆105Sep 5, 2023Updated 2 years ago
- ViennaRNA Package consists of a C code library for the prediction and comparison of RNA secondary structures☆15May 20, 2022Updated 4 years ago
- Python Repository of the Institute of Astronomy @ KU Leuven☆20Nov 5, 2020Updated 5 years ago
- Remote shuffle service for Apache Spark to store shuffle data on remote servers.☆335Sep 29, 2023Updated 2 years ago
- Mirror of Apache livy (Incubating)☆13Jul 7, 2026Updated 2 weeks ago
- Parquet file generator☆22Apr 17, 2018Updated 8 years ago
- Naos: Serialization-free RDMA networking in Java☆17Aug 17, 2021Updated 4 years ago
- Log driver plugin for docker explained. The boilerplate code here can also be used to write your own driver if you are feeling adventurou…☆13Mar 13, 2019Updated 7 years ago
- HDFS based on Java implementation as a remote ObjectStore for DataFusion☆10Feb 13, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Latest PASTE (NSDI'18) repository☆13May 2, 2022Updated 4 years ago
- Plugin to accelerate Spark SQL with the NEC Vector Engine.☆19Aug 15, 2022Updated 3 years ago
- Symbolic range analysis for LLVM.☆12Jan 10, 2016Updated 10 years ago
- template for https://cnli.me☆10Feb 27, 2025Updated last year
- On the fly, translation of Spark programs to run natively on your Oracle DB. Your Spark programs require no changes.☆36Apr 15, 2025Updated last year
- Symbolic Liveness Analysis of real-world software building upon KLEE to detect liveness violations (e.g. infinite loop bugs)☆12Dec 16, 2021Updated 4 years ago
- Client libraries of end users of Apache Kyuubi☆11May 15, 2026Updated 2 months ago