Spark* Shuffle plugin for support shuffling through remote persistent memory over fabrics, which leverages the RDMA network and remote persistent memory (for read) to provide extremely high performance and low latency shuffle solutions for Spark*.
☆14Sep 18, 2023Updated 3 years ago
Alternatives and similar repositories for pmem-shuffle
Users that are interested in pmem-shuffle are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Spark* shuffle plugin for support shuffling data through a remote Hadoop-compatible file system, as opposed to vanilla Spark's local-dis…☆21Mar 15, 2024Updated 2 years ago
- Source code for XPGraph-MICRO22☆11Apr 10, 2023Updated 3 years ago
- Spark Shuffle Optimization with RDMA+AEP☆30May 23, 2023Updated 3 years ago
- Spark* plug-in for accelerating Spark* SQL performance by using cache and index at SQL data source layer.☆37Jan 3, 2023Updated 3 years ago
- Tools for building, packaging, and OAP public cloud integrations such as AWS EMR, Google Dataproc and K8S.☆18Mar 27, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Apache Spark Connect Gateway☆15Updated this week
- Varka is a vectorized codegen for Spark SQL☆21Updated this week
- A package for predicting Rho independent terminators in bacterial genomes☆12Dec 14, 2015Updated 10 years ago
- Hadoop InputFormat for http://druid.io/☆10Oct 26, 2016Updated 10 years ago
- R package implementing the Nearest Neighbor Descent method for approximate nearest neighbors☆17Updated this week
- ☆12Apr 7, 2025Updated last year
- A Python library and command line utility for manipulating and plotting stellar lightcurves.☆10Jun 14, 2016Updated 10 years ago
- 基于多线程与epoll的高并发TCP服务器☆12Aug 4, 2018Updated 8 years ago
- Apache Spark - A unified analytics engine for large-scale data processing☆15Jul 24, 2023Updated 3 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Alerting and monitoring tool for Apache Spark☆23May 20, 2022Updated 4 years ago
- HMM-guided metagenomic gene-targeted assembler using iterative de Bruijn graphs☆18Oct 3, 2016Updated 10 years ago
- ☆18Dec 30, 2025Updated 10 months ago
- High Performance Network Library for RDMA☆28Jan 3, 2023Updated 3 years ago
- Firestorm is a Remote Shuffle Service, and provides the capability for Apache Spark and Apache Hadoop MapReduce applications to store shu…☆255Apr 7, 2023Updated 3 years ago
- ☆13Aug 9, 2024Updated 2 years ago
- Text Preprocessing in Python☆19Jan 15, 2017Updated 9 years ago
- Java implementation of the SCRAM SASL for both server and client plus examples☆17Apr 18, 2021Updated 5 years ago
- Passive Disaggregated Persistent Memory at USENIX ATC 2020.☆52Nov 2, 2020Updated 6 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- ☆12Mar 13, 2024Updated 2 years ago
- Remote shuffle service for Apache Spark to store shuffle data on remote servers.☆335Sep 29, 2023Updated 3 years ago
- Mirror of Apache livy (Incubating)☆13Jul 7, 2026Updated 3 months ago
- ViennaRNA Package consists of a C code library for the prediction and comparison of RNA secondary structures☆15May 20, 2022Updated 4 years ago
- benchmark-for-spark☆19May 7, 2025Updated last year
- Remote Persistent Memory Access Library☆105Sep 5, 2023Updated 3 years ago
- Python Repository of the Institute of Astronomy @ KU Leuven☆20Nov 5, 2020Updated 6 years ago
- A S3 Shuffle plugin for Apache Spark to enable elastic scaling for generic Spark workloads.☆52Sep 17, 2025Updated last year
- Naos: Serialization-free RDMA networking in Java☆17Aug 17, 2021Updated 5 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Log driver plugin for docker explained. The boilerplate code here can also be used to write your own driver if you are feeling adventurou…☆13Mar 13, 2019Updated 7 years ago
- A Python interface to gb-io, a fast GenBank parser written in Rust.☆25May 21, 2026Updated 5 months ago
- Parquet file generator☆22Apr 17, 2018Updated 8 years ago
- SOSP'21 Witcher Artifact☆10Aug 19, 2021Updated 5 years ago
- Plugin to accelerate Spark SQL with the NEC Vector Engine.☆19Aug 15, 2022Updated 4 years ago
- KLEE Symbolic Execution Engine☆10Jan 31, 2026Updated 9 months ago
- differential☆12Oct 30, 2019Updated 7 years ago