Must-read papers on improving efficiency for LLM serving clusters
☆34May 28, 2025Updated last year
Alternatives and similar repositories for Awesome-Large-Scale-LLM-Serving
Users that are interested in Awesome-Large-Scale-LLM-Serving are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This is a displayer for Live2Dv3 model☆19Mar 11, 2020Updated 6 years ago
- This is the implementation of the paper "Sparse Point Cloud Patches Rendering via Splitting 2D Gaussians"(CVPR 2025).☆21May 15, 2025Updated last year
- A curated reading list for machine learning reliability research and practice☆31Sep 18, 2025Updated 11 months ago
- Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction | A tiny BERT model can tell you the verbosity of an …☆52Jun 1, 2024Updated 2 years ago
- ☆12Jun 10, 2022Updated 4 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆12Oct 16, 2022Updated 3 years ago
- Code for the paper Alpha Zero in Continuous Action Space (A0C) (https://arxiv.org/pdf/1805.09613.pdf)☆15Jan 19, 2021Updated 5 years ago
- Code for ICRA2021 "Policy Transfer via Kinematic Domain Randomization and Adaptation"☆12Apr 28, 2021Updated 5 years ago
- Berkeley OS Prelim Reading Notes☆15Sep 20, 2023Updated 2 years ago
- A torch-based implementation of K-Means and K-Means++☆17Dec 6, 2020Updated 5 years ago
- Code for building ConceptNet from raw data.☆23Jan 26, 2024Updated 2 years ago
- Semantic-Aware Fine-Grained Correspondence, at ECCV 2022 (Oral)☆14Oct 29, 2022Updated 3 years ago
- 【云顶之弈小帮手】TFT-Helper☆15Jan 6, 2020Updated 6 years ago
- A development guide for building custom robot assemblies in Solidworks, converting them to URDF, importing to pybullet environment, and s…☆12Aug 25, 2020Updated 5 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading☆99Jun 16, 2025Updated last year
- ☆21Jun 9, 2025Updated last year
- PaiNN in jax☆11Jan 14, 2025Updated last year
- ☆22Aug 13, 2024Updated 2 years ago
- feishu custom bot action☆20Aug 14, 2026Updated last week
- 电动笛子！☆12Jul 13, 2024Updated 2 years ago
- Benchmark AFLOW Data Sets for Machine Learning doi.org/10.1007/s40192-020-00174-4☆11Aug 29, 2020Updated 5 years ago
- Automated Testing and Adaptive Detection of **Slow Faults** in Distributed Systems☆19Mar 6, 2025Updated last year
- ☆19May 9, 2025Updated last year
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- A verification tool for ensuring parallelization equivalence in distributed model training.☆18Sep 1, 2025Updated 11 months ago
- ☆15Jan 21, 2020Updated 6 years ago
- ☆14Mar 11, 2025Updated last year
- ☆19Jan 4, 2024Updated 2 years ago
- [COLM 2025] Official code for "When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoni…☆15Oct 31, 2025Updated 9 months ago
- DeepSeek-V3/R1 inference performance simulator☆194Mar 27, 2025Updated last year
- ☆13Jan 25, 2026Updated 6 months ago
- A compiler framework for eBPF programs☆22Jul 11, 2026Updated last month
- ☆25Dec 10, 2021Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Dataset generation for NeuralGrasps https://arxiv.org/abs/2207.02959☆24Sep 26, 2024Updated last year
- A benchmarking framework for on-device AI☆20Jul 30, 2026Updated 3 weeks ago
- [ICLR 2025] ELICIT: LLM Augmentation Via External In-context Capability☆14Mar 11, 2025Updated last year
- A rust-based benchmark for BlueField SmartNICs.☆30Jul 5, 2023Updated 3 years ago
- To assess the longtext capabilities more comprehensively, we propose Needle-in-a-Haystack PLUS, which shifts the focus from simple fact r…☆13Mar 4, 2024Updated 2 years ago
- [AAAI 2025] Neural-Symbolic Collaborative Distillation: Advancing Small Language Models for Complex Reasoning Tasks☆12Jun 19, 2025Updated last year
- a starter-kit for jaynes, the cloud-agnostic launch library☆17Apr 1, 2026Updated 4 months ago