Spark* Shuffle plugin for support shuffling through remote persistent memory over fabrics, which leverages the RDMA network and remote persistent memory (for read) to provide extremely high performance and low latency shuffle solutions for Spark*.
☆14Sep 18, 2023Updated 2 years ago
Alternatives and similar repositories for pmem-shuffle
Users that are interested in pmem-shuffle are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Spark* shuffle plugin for support shuffling data through a remote Hadoop-compatible file system, as opposed to vanilla Spark's local-dis…☆21Mar 15, 2024Updated 2 years ago
- Source code for XPGraph-MICRO22☆12Apr 10, 2023Updated 3 years ago
- Spark Shuffle Optimization with RDMA+AEP☆30May 23, 2023Updated 2 years ago
- Tools for building, packaging, and OAP public cloud integrations such as AWS EMR, Google Dataproc and K8S.☆18Mar 27, 2024Updated 2 years ago
- Spark* plug-in for accelerating Spark* SQL performance by using cache and index at SQL data source layer.☆37Jan 3, 2023Updated 3 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Hadoop InputFormat for http://druid.io/☆10Oct 26, 2016Updated 9 years ago
- ☆12Apr 7, 2025Updated last year
- 基于多线程与epoll的高并发TCP服务器☆11Aug 4, 2018Updated 7 years ago
- A Python library and command line utility for manipulating and plotting stellar lightcurves.☆10Jun 14, 2016Updated 9 years ago
- ☆12Mar 13, 2024Updated 2 years ago
- benchmark-for-spark☆18May 7, 2025Updated 11 months ago
- Alerting and monitoring tool for Apache Spark☆23May 20, 2022Updated 3 years ago
- Text Preprocessing in Python☆19Jan 15, 2017Updated 9 years ago
- A S3 Shuffle plugin for Apache Spark to enable elastic scaling for generic Spark workloads.☆52Sep 17, 2025Updated 6 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- High Performance Network Library for RDMA☆28Jan 3, 2023Updated 3 years ago
- Passive Disaggregated Persistent Memory at USENIX ATC 2020.☆51Nov 2, 2020Updated 5 years ago
- Firestorm is a Remote Shuffle Service, and provides the capability for Apache Spark and Apache Hadoop MapReduce applications to store shu…☆257Apr 7, 2023Updated 3 years ago
- Java implementation of the SCRAM SASL for both server and client plus examples☆17Apr 18, 2021Updated 4 years ago
- A Python interface to gb-io, a fast GenBank parser written in Rust.☆24Updated this week
- Apache Spark - A unified analytics engine for large-scale data processing☆16Jul 24, 2023Updated 2 years ago
- Parquet file generator☆22Apr 17, 2018Updated 7 years ago
- Remote shuffle service for Apache Spark to store shuffle data on remote servers.☆335Sep 29, 2023Updated 2 years ago
- Remote Persistent Memory Access Library☆105Sep 5, 2023Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- HMM-guided metagenomic gene-targeted assembler using iterative de Bruijn graphs☆18Oct 3, 2016Updated 9 years ago
- Mirror of Apache livy (Incubating)☆14Feb 11, 2026Updated 2 months ago
- Naos: Serialization-free RDMA networking in Java☆17Aug 17, 2021Updated 4 years ago
- ViennaRNA Package consists of a C code library for the prediction and comparison of RNA secondary structures☆15May 20, 2022Updated 3 years ago
- HDFS based on Java implementation as a remote ObjectStore for DataFusion☆10Feb 13, 2024Updated 2 years ago
- Plugin to accelerate Spark SQL with the NEC Vector Engine.☆19Aug 15, 2022Updated 3 years ago
- Latest PASTE (NSDI'18) repository☆13May 2, 2022Updated 3 years ago
- On the fly, translation of Spark programs to run natively on your Oracle DB. Your Spark programs require no changes.☆35Apr 15, 2025Updated last year
- Client libraries of end users of Apache Kyuubi☆11Jan 10, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Symbolic range analysis for LLVM.☆12Jan 10, 2016Updated 10 years ago
- template for https://cnli.me☆10Feb 27, 2025Updated last year
- Python Repository of the Institute of Astronomy @ KU Leuven☆20Nov 5, 2020Updated 5 years ago
- Symbolic Liveness Analysis of real-world software building upon KLEE to detect liveness violations (e.g. infinite loop bugs)☆12Dec 16, 2021Updated 4 years ago
- High-speed lossless data compression of 16 to 512 bytes--get better average compression than QuickLZ for 512-byte blocks. td512 maintains…☆27Feb 14, 2022Updated 4 years ago
- ☆11Aug 5, 2020Updated 5 years ago
- Log driver plugin for docker explained. The boilerplate code here can also be used to write your own driver if you are feeling adventurou…☆13Mar 13, 2019Updated 7 years ago