gty111 / gLLMLinks
gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
☆53Updated 2 weeks ago
Alternatives and similar repositories for gLLM
Users that are interested in gLLM are comparing it to the libraries listed below
Sorting:
- NVSHMEM‑Tutorial: Build a DeepEP‑like GPU Buffer☆154Updated 4 months ago
- DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit☆92Updated this week
- Multi-Level Triton Runner supporting Python, IR, PTX, and cubin.☆84Updated last week
- ☆87Updated 7 months ago
- A lightweight design for computation-communication overlap.☆213Updated last week
- Tile-based language built for AI computation across all scales☆117Updated 2 weeks ago
- [NeurIPS 2025] ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive☆66Updated last month
- ☆41Updated 2 months ago
- High performance Transformer implementation in C++.☆148Updated last year
- [HPCA 2026] A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆79Updated last month
- ☆112Updated 8 months ago
- Triton adapter for Ascend. Mirror of https://gitee.com/ascend/triton-ascend☆102Updated this week
- ☆41Updated 3 months ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆123Updated last month
- ☆93Updated 9 months ago
- a simple API to use CUPTI☆11Updated 5 months ago
- ☆32Updated last year
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆62Updated 10 months ago
- ☆65Updated 9 months ago
- PerFlow-AI is a programmable performance analysis, modeling, prediction tool for AI system.☆28Updated 3 weeks ago
- Canvas: End-to-End Kernel Architecture Search in Neural Networks☆27Updated last year
- DeepXTrace is a lightweight tool for precisely diagnosing slow ranks in DeepEP-based environments.☆90Updated 2 weeks ago
- Artifact of OSDI '24 paper, ”Llumnix: Dynamic Scheduling for Large Language Model Serving“☆64Updated last year
- From Minimal GEMM to Everything☆98Updated 3 weeks ago
- NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading☆79Updated 7 months ago
- Sequence-level 1F1B schedule for LLMs.☆38Updated 5 months ago
- A practical way of learning Swizzle☆36Updated 11 months ago
- Compiler for Dynamic Neural Networks☆45Updated 2 years ago
- ☆131Updated last year
- Accepted to MLSys 2026☆70Updated this week