Variant optimization autoscaler for distributed inference workloads
☆62Oct 13, 2026Updated 3 weeks ago
Alternatives and similar repositories for llm-d-autoscaling
Users that are interested in llm-d-autoscaling are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Simplified model deployment on llm-d☆28Jul 2, 2025Updated last year
- llm-d Router: The intelligent entry point for inference requests☆456Updated this week
- Inference Platform Simulation☆54Updated this week
- Model as a Service☆47Updated this week
- Distributed KV cache scheduling & offloading libraries☆191Oct 21, 2026Updated 2 weeks ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- ☆13Feb 11, 2025Updated last year
- GenAI inference performance benchmarking tool☆271Updated this week
- ☆33Oct 28, 2026Updated last week
- Holistic job manager on Kubernetes☆117Feb 20, 2024Updated 2 years ago
- llm-d benchmark scripts and tooling☆78Updated this week
- Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling☆290Updated this week
- Kubernetes controllers for fast model actuation using vLLM sleep/wake and launcher-based model swapping☆28Updated this week
- Scheduler-Plugins maintains a list of kubernetes scheduler plugins used in InftyAI community.☆11Jun 16, 2025Updated last year
- llm-d helm charts and deployment examples☆59May 1, 2026Updated 6 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- helm charts for deploying models with llm-d☆32Updated this week
- llm-d-batch-gateway is a standalone, backend-agnostic, OpenAI-compatible batch API and processing engine☆28Updated this week
- Engine-agnostic signal contract for model servers on Kubernetes: common metric semantics, engine profiles (e.g., vLLM, SGLang, TensorRT-L…☆18Updated this week
- InstaSlice Operator facilitates slicing of accelerators using stable APIs☆50Aug 11, 2026Updated 2 months ago
- Incubating P/D sidecar for llm-d☆17Nov 13, 2025Updated 11 months ago
- WG Serving☆38Mar 24, 2026Updated 7 months ago
- State of the art inference performance with modern accelerators on Kubernetes☆5,158Updated this week
- Gateway API Inference Extension☆801Updated this week
- A collection of useful Go libraries to ease the development of NVIDIA Operators for GPU/NIC management.☆30Oct 28, 2026Updated last week
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual h…☆233Updated this week
- ☆33Oct 8, 2026Updated 3 weeks ago
- ☆14May 11, 2026Updated 5 months ago
- Kubernetes APIServer 高性能代理组件，代理 APIServer 的 List 请求，其它类型的请求会直接反向代理到原生 APIServer。 CKube 还额外支持了分页、搜索和索引等功能。 并且，CKube 100% 兼容原生 kubectl 和 ku…☆19Sep 16, 2022Updated 4 years ago
- Operator for the mutating admission webhook for ClusterResourceOverride☆19Oct 15, 2026Updated 3 weeks ago
- SRO supports out-of-tree and third-party kernel drivers and the support software for the node OS via containers.☆16Aug 17, 2026Updated 2 months ago
- Let my Claude talk to yours.☆31Oct 1, 2026Updated last month
- Enable RHOAI User Workload Metrics for Single Serving Models☆10Oct 14, 2026Updated 3 weeks ago
- A workload for deploying LLM inference services on Kubernetes☆315Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆35Oct 30, 2026Updated last week
- Example DRA driver that developers can fork and modify to get them started writing their own.☆150Updated this week
- LeaderWorkerSet & DisaggregatedSet: Kubernetes APIs for Distributed and Disaggregated AI Serving☆902Updated this week
- Performance analysis for distributed inference systems☆15Updated this week
- Community maintained hardware plugin for vLLM on Spyre☆53Oct 1, 2026Updated last month
- 中国开发者活动日程（关注点：开源、开发者、云原生）☆30Updated this week
- Following the same workflows as Kubernetes. Widely used in InftyAI community.☆12Aug 17, 2026Updated 2 months ago