Infrawaves / DeepEP_ibrc_dual-ports_multiQPLinks
Aims to implement dual-port and multi-qp solutions in deepEP ibrc transport
☆47Updated 3 weeks ago
Alternatives and similar repositories for DeepEP_ibrc_dual-ports_multiQP
Users that are interested in DeepEP_ibrc_dual-ports_multiQP are comparing it to the libraries listed below
Sorting:
- ☆59Updated last month
- A lightweight design for computation-communication overlap.☆132Updated 3 weeks ago
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆47Updated 2 months ago
- FlexFlow Serve: Low-Latency, High-Performance LLM Serving☆39Updated 3 weeks ago
- High performance Transformer implementation in C++.☆124Updated 4 months ago
- A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆36Updated last month
- [DAC'25] Official implement of "HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference"☆50Updated 2 weeks ago
- DeeperGEMM: crazy optimized version☆69Updated 3 weeks ago
- ☆73Updated 2 weeks ago
- NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading☆35Updated 3 months ago
- ☆41Updated this week
- ⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.☆79Updated 3 weeks ago
- ☆49Updated 2 weeks ago
- A practical way of learning Swizzle☆19Updated 4 months ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆81Updated 2 weeks ago
- Ultra | Ultimate | Unified CCL☆87Updated this week
- ☆25Updated 3 months ago
- ☆65Updated 2 months ago
- Scalable long-context LLM decoding that leverages sparsity—by treating the KV cache as a vector storage system.☆41Updated last week
- ☆73Updated 4 months ago
- Implement Flash Attention using Cute.☆85Updated 5 months ago
- Artifact of OSDI '24 paper, ”Llumnix: Dynamic Scheduling for Large Language Model Serving“☆61Updated 11 months ago
- Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.☆36Updated 2 months ago
- ☆62Updated 11 months ago
- ☆42Updated this week
- Artifact for "Marconi: Prefix Caching for the Era of Hybrid LLMs" [MLSys '25 Outstanding Paper Honorable Mention]☆10Updated 2 months ago
- 使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention☆68Updated 9 months ago
- ☆76Updated last month
- Tacker: Tensor-CUDA Core Kernel Fusion for Improving the GPU Utilization while Ensuring QoS☆25Updated 3 months ago
- Thunder Research Group's Collective Communication Library☆37Updated last year