Kubernetes controllers for fast model actuation using vLLM sleep/wake and launcher-based model swapping
☆28Oct 24, 2026Updated this week
Alternatives and similar repositories for llm-d-fast-model-actuation
Users that are interested in llm-d-fast-model-actuation are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Following the same workflows as Kubernetes. Widely used in InftyAI community.☆12Aug 17, 2026Updated 2 months ago
- Incubating P/D sidecar for llm-d☆17Nov 13, 2025Updated 11 months ago
- llm-d Router: The intelligent entry point for inference requests☆432Updated this week
- ☆12May 22, 2023Updated 3 years ago
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆13May 16, 2023Updated 3 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- llm-d-batch-gateway is a standalone, backend-agnostic, OpenAI-compatible batch API and processing engine☆28Updated this week
- 💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…☆27Dec 6, 2024Updated last year
- d.run website☆18Oct 21, 2026Updated last week
- Simplified Data Management and Sharing for Kubernetes☆18Updated this week
- ☆13Feb 11, 2025Updated last year
- This project is designed to simulate GPU information, making it easier to test scenarios where a GPU is not available.☆65Jan 9, 2026Updated 9 months ago
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆35Oct 22, 2026Updated last week
- Kubernetes APIServer 高性能代理组件，代理 APIServer 的 List 请求，其它类型的请求会直接反向代理到原生 APIServer。 CKube 还额外支持了分页、搜索和索引等功能。 并且，CKube 100% 兼容原生 kubectl 和 ku…☆19Sep 16, 2022Updated 4 years ago
- Scheduler-Plugins maintains a list of kubernetes scheduler plugins used in InftyAI community.☆11Jun 16, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- llm-d benchmark scripts and tooling☆76Updated this week
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆27Apr 24, 2025Updated last year
- A workload for deploying LLM inference services on Kubernetes☆312Updated this week
- Variant optimization autoscaler for distributed inference workloads☆61Oct 13, 2026Updated 2 weeks ago
- Nano vLLM☆13Jun 26, 2025Updated last year
- C-ray — X-ray your containers. See beneath the runtime.☆23Jun 25, 2026Updated 4 months ago
- WG Serving☆38Mar 24, 2026Updated 7 months ago
- Distributed KV cache scheduling & offloading libraries☆187Oct 21, 2026Updated last week
- GenAI inference performance benchmarking tool☆266Oct 22, 2026Updated last week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling☆280Updated this week
- 中国开发者活动日程（关注点： 开源、开发者、云原生）☆30Oct 21, 2026Updated last week
- Holistic job manager on Kubernetes☆117Feb 20, 2024Updated 2 years ago
- Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and i…☆195Updated this week
- Helm charts for llm-d☆52Jul 22, 2025Updated last year
- helm charts for deploying models with llm-d☆32Oct 20, 2026Updated last week
- Discover ingress-nginx usage and auto-generate Gateway API migration plans before ingress-nginx reaches end-of-life (March 2026).☆16Nov 26, 2025Updated 11 months ago
- ☆11Jul 6, 2023Updated 3 years ago
- Parse Dat protocol SLEEP files.☆13Apr 28, 2021Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- FastAPI server for hosting Guardrails AI in your own environment.☆17Updated this week
- Node-level enforcement and metrics for fractional GPU allocations made by KAI Scheduler.☆48Updated this week
- Provides deploy scripts and CSI for Lustre.☆14Updated this week
- Let CI Robot automatically execute commands for your PR/issue in your Github repository, hosting on Github Action does not require your s…☆13Oct 19, 2026Updated last week
- AgentX Harness - Realistic 1mil Long Context, Multi Turn Agentic Workload Trace Replayer, collected from over $3mil USD of Traces | Agent…☆36Updated this week
- Validation and management tools for NVIDIA AI Cloud environments.☆34Updated this week
- A set of common libraries for writing KubeVela ecosystem controllers.☆12Oct 19, 2026Updated last week