Dist-MoE implements Speed-of-Light distributed Mixture-of-Experts forward and backward kernels. They are highly optimized and fused comm-compute kernels for training and inference on SM100+ GPUs
☆17Oct 17, 2026Updated this week
Alternatives and similar repositories for dist_moe
Users that are interested in dist_moe are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Ulysses sequence-parallel all-to-all as a torch custom op, moved by the GPU copy engines into torch symmetric memory. Zero SM usage; 1.66…☆54Aug 25, 2026Updated last month
- NaiveRT: a self-contained TP8 inference runtime for the Naive N series on eight Blackwell GPUs with UMMA support. One persistent CUDA gra…☆30Sep 27, 2026Updated 3 weeks ago
- ☆57Sep 27, 2026Updated 3 weeks ago
- Curated high-performance TIRx GPU kernels☆32Updated this week
- ☆38Aug 4, 2026Updated 2 months ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Experiments, results, and misc. scripts.☆31Oct 2, 2026Updated 2 weeks ago
- Communication patterns for AI, built on top of NCCL device and host APIs☆69Updated this week
- A collection of megakernels for TPU☆165Updated this week
- ☆46Aug 3, 2026Updated 2 months ago
- Tutti is a GPU-centric KV Cache Store to make SSD work as DRAM.☆64Oct 14, 2026Updated last week
- https://arxiv.org/abs/2610.13990☆30Updated this week
- Efficient Mask-Aware Distributed Attention for Long-Context Training☆50Updated this week
- ☆116Aug 5, 2026Updated 2 months ago
- DeepStack: Facilitating Co-Design Exploration of 3D DRAM-Stacked Accelerators for Distributed LLM Inference. Includes the MICRO 2026 AE a…☆65Oct 5, 2026Updated 2 weeks ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- DeepSelect: TopK kernels for DeepSeek Sparse Attention (DSA) and Samplers☆495Sep 30, 2026Updated 3 weeks ago
- TensorIR is a lightweight NVIDIA-owned MLIR compiler frontend for expressing tensor computations and lowering them to NVIDIA CUDA Tile IR…☆140Updated this week
- Performance benchmark & accuracy evaluation for vLLM☆17Updated this week
- A linker for partial GPU kernels.☆28Oct 12, 2026Updated last week
- ☆37Aug 7, 2025Updated last year
- A high-performance, RDMA-native data-plane library for large-scale heterogeneous memory pools, developed by the Tencent Astral Network Te…☆32Aug 6, 2026Updated 2 months ago
- Lightweight Benchmark Server for Agentic GPU Programming☆58Updated this week
- ☆11Jun 9, 2023Updated 3 years ago
- An open, visual atlas of looped transformers, recurrent computation, adaptive depth, and latent reasoning.☆33Sep 26, 2026Updated 3 weeks ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Synchronizing Claude Code conversations across machines☆16Sep 30, 2026Updated 2 weeks ago
- ☆46Oct 15, 2025Updated last year
- Production-ready MoE load balancing via real-time expert replication☆259Updated this week
- MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts☆1,186Oct 12, 2026Updated last week
- KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels☆53Jun 1, 2026Updated 4 months ago
- DeeperGEMM: crazy optimized version☆85May 5, 2025Updated last year
- Reference implementation and examples of the CuTe Layout representation and algebra.☆379Sep 30, 2026Updated 2 weeks ago
- Handwritten GEMM using Intel AMX (Advanced Matrix Extension)☆17Jan 11, 2025Updated last year
- ☆16Nov 14, 2023Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Matrix multiplication on GPUs for matrices stored on a CPU. Similar to cublasXt, but ported to both NVIDIA and AMD GPUs.☆33Apr 2, 2025Updated last year
- TiledLower is a Dataflow Analysis and Codegen Framework written in Rust.☆13Nov 23, 2024Updated last year
- ☆68Feb 24, 2026Updated 7 months ago
- Sample Codes using NVSHMEM on Multi-GPU☆30Jan 22, 2023Updated 3 years ago
- A lightweight library for xPU kernel JIT compilation☆399Updated this week
- Unified tracing profiler and visualizer, one timeline from CPU to GPU/HES to in-kernel zones☆149Sep 21, 2026Updated 3 weeks ago
- NVSHMEM‑Tutorial: Build a DeepEP‑like GPU Buffer☆207Feb 11, 2026Updated 8 months ago