DeepAuto-AI / sglangLinks
This is a fork of SGLang for hip-attention integration. Please refer to hip-attention for detail.
☆17Updated last week
Alternatives and similar repositories for sglang
Users that are interested in sglang are comparing it to the libraries listed below
Sorting:
- Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.☆148Updated this week
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆128Updated 10 months ago
- [NeurIPS'25 Oral] Query-agnostic KV cache eviction: 3–4× reduction in memory and 2× decrease in latency (Qwen3/2.5, Gemma3, LLaMA3)☆111Updated last week
- Beyond KV Caching: Shared Attention for Efficient LLMs☆19Updated last year
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- Boosting 4-bit inference kernels with 2:4 Sparsity☆82Updated last year
- ☆52Updated 11 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated 11 months ago
- QuIP quantization☆60Updated last year
- ☆86Updated 8 months ago
- KV cache compression for high-throughput LLM inference☆141Updated 8 months ago
- Work in progress.☆74Updated 3 months ago
- ☆98Updated last month
- The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.☆49Updated 11 months ago
- My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated☆33Updated last year
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- Transformers components but in Triton☆34Updated 4 months ago
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆118Updated 3 months ago
- ☆142Updated 7 months ago
- ☆57Updated last year
- RWKV-7: Surpassing GPT☆96Updated 10 months ago
- A repository for research on medium sized language models.☆78Updated last year
- Official implementation for Training LLMs with MXFP4☆93Updated 5 months ago
- ☆27Updated last year
- Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)☆161Updated 5 months ago
- ☆20Updated 5 months ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆168Updated last year
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆43Updated 3 months ago
- ☆38Updated 11 months ago
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆48Updated 5 months ago