Must-read papers on improving efficiency for LLM serving clusters
☆34May 28, 2025Updated last year
Alternatives and similar repositories for Awesome-Large-Scale-LLM-Serving
Users that are interested in Awesome-Large-Scale-LLM-Serving are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆39Mar 17, 2025Updated last year
- A curated reading list for machine learning reliability research and practice☆31Sep 18, 2025Updated 10 months ago
- Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction | A tiny BERT model can tell you the verbosity of an …☆52Jun 1, 2024Updated 2 years ago
- Calibrating LLM Confidence by Probing Perturbed Representation Stability☆19Jul 5, 2025Updated last year
- ☆12Oct 16, 2022Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ICSE2021 Submission☆13Aug 28, 2022Updated 3 years ago
- ☆20Aug 26, 2021Updated 4 years ago
- 🧠Plan-and-Budget: Training-free test-time reasoning framework for adaptive token allocation in large language models (ICLR 2026).☆15Mar 2, 2026Updated 4 months ago
- 为GPT/GLM等LLM大语言模型提供实用化交互接口，特别优化论文阅读/润色/写作体验，模块化设计，支持自定义快捷按钮&函数插件，支持Python和C++等项目剖析&自译解功能，PDF/LaTex论文翻译&总结功能，支持并行问询多种LLM模型，支持chatglm3等本地模型…☆15Feb 18, 2024Updated 2 years ago
- NLPCC2023 shared-task DiaASQ first-place solution. （NLPCC2023对话式细粒度情感识别大赛第一名方案）☆15Jun 21, 2023Updated 3 years ago
- Repository for GeoUni, A Unified Model for Generating Geometry Diagrams, Problems and Problem Solutions.☆23Jun 12, 2025Updated last year
- A torch-based implementation of K-Means and K-Means++☆17Dec 6, 2020Updated 5 years ago
- 【云顶之弈小帮手】TFT-Helper☆15Jan 6, 2020Updated 6 years ago
- [NeurIPS 2024] Can Language Models Learn to Skip Steps?☆21Jan 25, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- The repository for papaer "Distance between Relevant Information Pieces Causes Bias in Long-Context LLMs"☆14Dec 16, 2024Updated last year
- NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading☆100Jun 16, 2025Updated last year
- another BibTeX-to-HTML (with Jinja2 templates)☆17Jun 23, 2020Updated 6 years ago
- PaiNN in jax☆11Jan 14, 2025Updated last year
- Benchmark AFLOW Data Sets for Machine Learning doi.org/10.1007/s40192-020-00174-4☆11Aug 29, 2020Updated 5 years ago
- 电动笛子！☆12Jul 13, 2024Updated 2 years ago
- Automated Testing and Adaptive Detection of **Slow Faults** in Distributed Systems☆19Mar 6, 2025Updated last year
- Open-Pandora: On-the-fly Control Video Generation☆35Nov 28, 2024Updated last year
- ☆19May 9, 2025Updated last year
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- A verification tool for ensuring parallelization equivalence in distributed model training.☆18Sep 1, 2025Updated 11 months ago
- ☆14Mar 11, 2025Updated last year
- [COLM 2025] Official code for "When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoni…☆15Oct 31, 2025Updated 9 months ago
- An Observability Framework for AI Training☆73Jul 16, 2026Updated 2 weeks ago
- A compiler framework for eBPF programs☆21Jul 11, 2026Updated 3 weeks ago
- A benchmarking framework for on-device AI☆19Updated this week
- 🫧 Code for Holistic Reasoning with Long-Context LMs: A Benchmark for Database Operations on Massive Textual Data (Maekawa*, Iso* et al.…☆12Feb 25, 2025Updated last year
- ☆38Jun 3, 2023Updated 3 years ago
- ☆12Apr 29, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- [ICLR 2025] ELICIT: LLM Augmentation Via External In-context Capability☆14Mar 11, 2025Updated last year
- ☆45Apr 2, 2023Updated 3 years ago
- Code for evaluating WebRTC performance☆14Sep 18, 2017Updated 8 years ago
- [AAAI 2025] Neural-Symbolic Collaborative Distillation: Advancing Small Language Models for Complex Reasoning Tasks☆12Jun 19, 2025Updated last year
- [ASPLOS' 26] TetriServe: Efficiently Serving Mixed DiT Workloads☆17Mar 12, 2026Updated 4 months ago
- ☆14Oct 3, 2024Updated last year
- Confidence Regulation Neurons in Language Models (NeurIPS 2024)☆15Feb 1, 2025Updated last year