This is archive of SparkRDMA project. The new repository with RDMA shuffle acceleration for Apache Spark is here: https://github.com/Nvidia/sparkucx
☆257May 13, 2019Updated 6 years ago
Alternatives and similar repositories for SparkRDMA
Users that are interested in SparkRDMA are comparing it to the libraries listed below
Sorting:
- DiSNI: Direct Storage and Networking Interface☆194Mar 9, 2023Updated 3 years ago
- Mirror of Apache crail (Incubating)☆151Jul 3, 2022Updated 3 years ago
- Fast I/O plugins for Spark☆41Dec 14, 2020Updated 5 years ago
- DaRPC: Data Center Remote Procedure Call☆56Oct 13, 2020Updated 5 years ago
- Spark Shuffle Optimization with RDMA+AEP☆30May 23, 2023Updated 2 years ago
- A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer☆32Jun 5, 2024Updated last year
- A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer☆52Oct 30, 2023Updated 2 years ago
- Spark Terasort☆121Apr 21, 2023Updated 2 years ago
- Naos: Serialization-free RDMA networking in Java☆17Aug 17, 2021Updated 4 years ago
- A framework to understand RDMA☆409Oct 12, 2023Updated 2 years ago
- Java API over Accelio☆84Jan 15, 2017Updated 9 years ago
- Linux user space library for network socket acceleration based on RDMA compatible network adaptors☆679Feb 8, 2026Updated last month
- Splash, a flexible Spark shuffle manager that supports user-defined storage backends for shuffle data storage and exchange☆130Dec 19, 2024Updated last year
- LITE Kernel RDMA Support for Datacenter Applications. SOSP 2017.☆111Jul 9, 2020Updated 5 years ago
- RDMA Large File Transfer：A Large-scale File and Folder Transfer Framework. By using two components, RDMA Channel and RDMA Verbs.☆14Jan 9, 2020Updated 6 years ago
- A tutorial on RDMA based programming using code examples☆603Jan 3, 2020Updated 6 years ago
- ☆68May 1, 2017Updated 8 years ago
- Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group)☆1,583Updated this week
- Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.☆257Feb 21, 2023Updated 3 years ago
- A NVMf library for Java☆30Aug 15, 2019Updated 6 years ago
- High Performance Network Library for RDMA☆28Jan 3, 2023Updated 3 years ago
- Fast In-memory Transaction Processing using Hybrid RDMA Primitives☆66Nov 15, 2018Updated 7 years ago
- OFED libibverbs tests package☆16Oct 5, 2021Updated 4 years ago
- Kubernetes deployment of PrestoDB, Hive Metastore, and Minio S3-standard object store☆17Oct 20, 2022Updated 3 years ago
- JVM integration for Weld☆16Sep 24, 2018Updated 7 years ago
- RDMA core userspace libraries and daemons☆2,146Feb 25, 2026Updated last week
- Use the TPC-DS benchmark to test Spark SQL performance☆184Apr 27, 2020Updated 5 years ago
- High performance data store solution☆1,446Mar 1, 2026Updated last week
- Enhanced networking support for TensorFlow. Maintained by SIG-networking.☆98Nov 19, 2021Updated 4 years ago
- Ansible playbook for automated HDP 2.x deployment install with Kerberos☆19Sep 8, 2016Updated 9 years ago
- Performance Analysis Tool☆78Nov 25, 2025Updated 3 months ago
- ☆76Sep 13, 2016Updated 9 years ago
- Infiniswap enables unmodified applications to efficiently use disaggregated memory.☆256Sep 26, 2020Updated 5 years ago
- Smart Storage Management for Big Data, a comprehensive hot/cold data optimized solution☆140Jan 3, 2023Updated 3 years ago
- Helm Chart for lyft/flinkk8soperator☆11Mar 10, 2020Updated 5 years ago
- Demonstration of flow control over RDMA fabric☆13Jun 28, 2018Updated 7 years ago
- Tensorflow is a computational library using data flow graphs for scalable machine learning, and Tensorflow-RDMA is the implementation ov…☆59Nov 27, 2022Updated 3 years ago
- Infiniband Verbs Performance Tests☆919Mar 2, 2026Updated last week
- Spark metrics related custom classes and sinks (e.g. Prometheus)☆188Aug 2, 2022Updated 3 years ago