flexflow / flexflow-serveLinks
FlexFlow Serve: Low-Latency, High-Performance LLM Serving
☆66Updated 3 months ago
Alternatives and similar repositories for flexflow-serve
Users that are interested in flexflow-serve are comparing it to the libraries listed below
Sorting:
- A lightweight design for computation-communication overlap.☆209Updated 2 weeks ago
- ☆81Updated 2 months ago
- High performance Transformer implementation in C++.☆148Updated 11 months ago
- ☆130Updated last year
- ☆92Updated 9 months ago
- NVSHMEM‑Tutorial: Build a DeepEP‑like GPU Buffer☆152Updated 3 months ago
- Stateful LLM Serving☆93Updated 10 months ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆123Updated 2 weeks ago
- Tile-based language built for AI computation across all scales☆115Updated 3 weeks ago
- [HPCA 2026] A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆76Updated 3 weeks ago
- [NeurIPS 2025] ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive☆62Updated last month
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity☆231Updated 2 years ago
- Microsoft Collective Communication Library☆66Updated last year
- Official repository for the paper DynaPipe: Optimizing Multi-task Training through Dynamic Pipelines☆19Updated 2 years ago
- Aims to implement dual-port and multi-qp solutions in deepEP ibrc transport☆73Updated 8 months ago
- DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit☆91Updated last week
- PyTorch library for cost-effective, fast and easy serving of MoE models.☆273Updated 2 months ago
- Artifact of OSDI '24 paper, ”Llumnix: Dynamic Scheduling for Large Language Model Serving“☆64Updated last year
- [ICML 2024] Serving LLMs on heterogeneous decentralized clusters.☆34Updated last year
- Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.☆69Updated 9 months ago
- ☆338Updated last week
- Autonomous GPU Kernel Generation via Deep Agents☆202Updated this week
- High-performance distributed data shuffling (all-to-all) library for MoE training and inference☆106Updated 2 weeks ago
- Thunder Research Group's Collective Communication Library☆46Updated 6 months ago
- ☆65Updated 8 months ago
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆134Updated last year
- ☆104Updated last year
- ☆84Updated 3 years ago
- ☆162Updated 5 months ago
- LLM serving cluster simulator☆132Updated last year