Ketonmi / Awesome-Large-Scale-LLM-ServingLinks
Must-read papers on improving efficiency for LLM serving clusters
β31Updated 3 months ago
Alternatives and similar repositories for Awesome-Large-Scale-LLM-Serving
Users that are interested in Awesome-Large-Scale-LLM-Serving are comparing it to the libraries listed below
Sorting:
- π° Must-read papers on KV Cache Compression (constantly updating π€).β540Updated last month
- Awesome-LLM-KV-Cache: A curated list of πAwesome LLM KV Cache Papers with Codes.β364Updated 6 months ago
- π° Must-read papers and blogs on Speculative Decoding β‘οΈβ938Updated last week
- Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)β313Updated 5 months ago
- β137Updated 2 months ago
- This repository serves as a comprehensive survey of LLM development, featuring numerous research papers along with their corresponding coβ¦β205Updated last month
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rankβ59Updated 10 months ago
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papβ¦β270Updated 6 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferenceβ335Updated 2 months ago
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Lengthβ113Updated 5 months ago
- β26Updated 6 months ago
- β608Updated 4 months ago
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of β¦β264Updated 3 months ago
- Curated collection of papers in MoE model inferenceβ265Updated last week
- a curated list of high-quality papers on resource-efficient LLMs π±β139Updated 6 months ago
- β143Updated 2 months ago
- paper and its code for AI Systemβ327Updated last month
- A large-scale simulation framework for LLM inferenceβ443Updated 2 months ago
- Code repo for efficient quantized MoE inference with mixture of low-rank compensatorsβ24Updated 5 months ago
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)β154Updated last year
- Summary of some awesome work for optimizing LLM inferenceβ110Updated 3 months ago
- [NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.β475Updated last year
- β58Updated 9 months ago
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.β404Updated this week
- PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".β90Updated 2 years ago
- Systems for GenAIβ143Updated 5 months ago
- PyTorch library for cost-effective, fast and easy serving of MoE models.β238Updated 2 months ago
- Fast inference from large lauguage models via speculative decodingβ827Updated last year
- Super-Efficient RLHF Training of LLMs with Parameter Reallocationβ317Updated 5 months ago
- β130Updated 11 months ago