inclusionAI / asystem-astateLinks
☆30Updated last month
Alternatives and similar repositories for asystem-astate
Users that are interested in asystem-astate are comparing it to the libraries listed below
Sorting:
- A NCCL extension library, designed to efficiently offload GPU memory allocated by the NCCL communication library.☆77Updated 3 weeks ago
- Research prototype of PRISM — a cost-efficient multi-LLM serving system with flexible time- and space-based GPU sharing.☆51Updated 4 months ago
- ☆72Updated 3 months ago
- DeepXTrace is a lightweight tool for precisely diagnosing slow ranks in DeepEP-based environments.☆83Updated 2 weeks ago
- Stateful LLM Serving☆93Updated 9 months ago
- Scalable long-context LLM decoding that leverages sparsity—by treating the KV cache as a vector storage system.☆108Updated last week
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆256Updated last month
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank☆66Updated last year
- ☆81Updated 2 months ago
- ☆337Updated this week
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆204Updated last year
- [NeurIPS 2025] ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive☆60Updated 3 weeks ago
- Preview Code for Continuum Paper☆19Updated this week
- ☆130Updated last year
- ☆160Updated 5 months ago
- A framework for generating realistic LLM serving workloads☆94Updated 3 months ago
- ☆70Updated 2 months ago
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆133Updated last year
- KV cache store for distributed LLM inference☆378Updated last month
- A lightweight design for computation-communication overlap.☆207Updated 2 weeks ago
- High performance Transformer implementation in C++.☆148Updated 11 months ago
- ☆73Updated last year
- Nex Venus Communication Library☆70Updated last month
- ☆79Updated 3 years ago
- Automated Parallelization System and Infrastructure for Multiple Ecosystems☆82Updated last year
- ☆19Updated 6 months ago
- Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding☆74Updated last month
- Utility scripts for PyTorch (e.g. Make Perfetto show some disappearing kernels, Memory profiler that understands more low-level allocatio…☆73Updated 3 months ago
- High-performance distributed data shuffling (all-to-all) library for MoE training and inference☆102Updated last week
- ☆47Updated last year