Summary of some awesome work for optimizing LLM inference
☆278Feb 14, 2026Updated 9 months ago
Alternatives and similar repositories for LLM-inference-optimization-paper
Users that are interested in LLM-inference-optimization-paper are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆285Mar 6, 2025Updated last year
- ☆33Sep 26, 2025Updated last year
- Large Language Model (LLM) Systems Paper List☆2,328Oct 20, 2026Updated 3 weeks ago
- 📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉☆5,565Oct 20, 2026Updated 3 weeks ago
- Curated collection of papers in machine learning systems☆663Oct 31, 2026Updated 2 weeks ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Curated collection of papers in MoE model inference☆434Mar 12, 2026Updated 8 months ago
- This repository serves as a comprehensive survey of LLM development, featuring numerous research papers along with their corresponding co…☆374Jul 16, 2026Updated 4 months ago
- paper and its code for AI System☆382May 14, 2026Updated 6 months ago
- Disaggregated serving system for Large Language Models (LLMs).☆847Apr 6, 2025Updated last year
- Artifact of Chimera☆18May 6, 2025Updated last year
- LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure☆473Nov 8, 2026Updated last week
- ☆15Jun 26, 2024Updated 2 years ago
- Dynamic Memory Management for Serving LLMs without PagedAttention☆528Aug 24, 2026Updated 2 months ago
- Accurate, large-scale, and extensible simulator for LLM inference Systems☆731Aug 24, 2026Updated 2 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Artifact for "Marconi: Prefix Caching for the Era of Hybrid LLMs" [MLSys '25 Outstanding Paper Award, Honorable Mention]☆74Mar 5, 2025Updated last year
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆228Oct 14, 2026Updated last month
- A low-latency & high-throughput serving engine for LLMs☆538Updated this week
- ☆32May 28, 2024Updated 2 years ago
- Efficient and easy multi-instance LLM serving☆561Mar 12, 2026Updated 8 months ago
- NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading☆101Jun 16, 2025Updated last year
- codebase for MUSTAFAR:Promoting Unstructured Sparsity for KV Pruning in LLM Inference☆12Nov 6, 2025Updated last year
- ☆136Nov 11, 2024Updated 2 years ago
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity☆246Sep 24, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction | A tiny BERT model can tell you the verbosity of an …☆53Jun 1, 2024Updated 2 years ago
- [MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Se…☆869Mar 6, 2025Updated last year
- FlashInfer: Kernel Library for LLM Serving☆6,949Updated this week
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.☆7,061Updated this week
- Paper reading and discussion notes, covering AI frameworks, distributed systems, cluster management, etc.☆78Mar 4, 2026Updated 8 months ago
- ☆648Jan 14, 2026Updated 10 months ago
- An Open-source book with Modern CUDA learning Notes for Beginners - BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.☆12,187Updated this week
- Latency and Memory Analysis of Transformer Models for Training and Inference☆497Apr 19, 2025Updated last year
- A throughput-oriented high-performance serving framework for LLMs☆978Mar 29, 2026Updated 7 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆366Jun 10, 2025Updated last year
- ☆54Apr 29, 2025Updated last year
- Materials for learning SGLang☆905Jan 5, 2026Updated 10 months ago
- Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline mod…☆693Sep 11, 2024Updated 2 years ago
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank☆85Nov 4, 2024Updated 2 years ago
- This repository contains the code for the ICLR 2026 paper “DASH: Deterministic Attention Scheduling for High-Throughput Reproducible LLM …☆15Jun 23, 2026Updated 4 months ago
- ☆73Oct 14, 2025Updated last year
Are these results useful?