Spark* Shuffle plugin for support shuffling through remote persistent memory over fabrics, which leverages the RDMA network and remote persistent memory (for read) to provide extremely high performance and low latency shuffle solutions for Spark*.
☆14Sep 18, 2023Updated 2 years ago
Alternatives and similar repositories for pmem-shuffle
Users that are interested in pmem-shuffle are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Spark* shuffle plugin for support shuffling data through a remote Hadoop-compatible file system, as opposed to vanilla Spark's local-dis…☆21Mar 15, 2024Updated 2 years ago
- Source code for XPGraph-MICRO22☆12Apr 10, 2023Updated 2 years ago
- Spark Shuffle Optimization with RDMA+AEP☆30May 23, 2023Updated 2 years ago
- Tools for building, packaging, and OAP public cloud integrations such as AWS EMR, Google Dataproc and K8S.☆18Mar 27, 2024Updated 2 years ago
- Spark* plug-in for accelerating Spark* SQL performance by using cache and index at SQL data source layer.☆37Jan 3, 2023Updated 3 years ago
- NordVPN Special Discount Offer • AdSave on top-rated NordVPN 1 or 2-year plans with secure browsing, privacy protection, and support for for all major platforms.
- Hadoop InputFormat for http://druid.io/☆10Oct 26, 2016Updated 9 years ago
- ☆12Apr 7, 2025Updated 11 months ago
- 基于多线程与epoll的高并发TCP服务器☆11Aug 4, 2018Updated 7 years ago
- A Python library and command line utility for manipulating and plotting stellar lightcurves.☆10Jun 14, 2016Updated 9 years ago
- ☆12Mar 13, 2024Updated 2 years ago
- benchmark-for-spark☆18May 7, 2025Updated 10 months ago
- Alerting and monitoring tool for Apache Spark☆23May 20, 2022Updated 3 years ago
- Text Preprocessing in Python☆19Jan 15, 2017Updated 9 years ago
- A S3 Shuffle plugin for Apache Spark to enable elastic scaling for generic Spark workloads.☆52Sep 17, 2025Updated 6 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- High Performance Network Library for RDMA☆28Jan 3, 2023Updated 3 years ago
- Passive Disaggregated Persistent Memory at USENIX ATC 2020.☆51Nov 2, 2020Updated 5 years ago
- Firestorm is a Remote Shuffle Service, and provides the capability for Apache Spark and Apache Hadoop MapReduce applications to store shu…☆257Apr 7, 2023Updated 2 years ago
- Java implementation of the SCRAM SASL for both server and client plus examples☆17Apr 18, 2021Updated 4 years ago
- A Python interface to gb-io, a fast GenBank parser written in Rust.☆24Mar 9, 2026Updated 2 weeks ago
- Apache Spark - A unified analytics engine for large-scale data processing☆16Jul 24, 2023Updated 2 years ago
- Parquet file generator☆22Apr 17, 2018Updated 7 years ago
- Remote shuffle service for Apache Spark to store shuffle data on remote servers.☆335Sep 29, 2023Updated 2 years ago
- Remote Persistent Memory Access Library☆106Sep 5, 2023Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- HMM-guided metagenomic gene-targeted assembler using iterative de Bruijn graphs☆18Oct 3, 2016Updated 9 years ago
- Mirror of Apache livy (Incubating)☆14Feb 11, 2026Updated last month
- Naos: Serialization-free RDMA networking in Java☆17Aug 17, 2021Updated 4 years ago
- ViennaRNA Package consists of a C code library for the prediction and comparison of RNA secondary structures☆15May 20, 2022Updated 3 years ago
- HDFS based on Java implementation as a remote ObjectStore for DataFusion☆10Feb 13, 2024Updated 2 years ago
- Plugin to accelerate Spark SQL with the NEC Vector Engine.☆19Aug 15, 2022Updated 3 years ago
- Latest PASTE (NSDI'18) repository☆13May 2, 2022Updated 3 years ago
- On the fly, translation of Spark programs to run natively on your Oracle DB. Your Spark programs require no changes.☆35Apr 15, 2025Updated 11 months ago
- Client libraries of end users of Apache Kyuubi☆11Jan 10, 2023Updated 3 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Symbolic range analysis for LLVM.☆12Jan 10, 2016Updated 10 years ago
- template for https://cnli.me☆10Feb 27, 2025Updated last year
- Python Repository of the Institute of Astronomy @ KU Leuven☆20Nov 5, 2020Updated 5 years ago
- Symbolic Liveness Analysis of real-world software building upon KLEE to detect liveness violations (e.g. infinite loop bugs)☆12Dec 16, 2021Updated 4 years ago
- High-speed lossless data compression of 16 to 512 bytes--get better average compression than QuickLZ for 512-byte blocks. td512 maintains…☆27Feb 14, 2022Updated 4 years ago
- ☆11Aug 5, 2020Updated 5 years ago
- Log driver plugin for docker explained. The boilerplate code here can also be used to write your own driver if you are feeling adventurou…☆13Mar 13, 2019Updated 7 years ago