llm-d / llm-d-kv-cache-manager
Distributed KV cache coordinator
☆31Updated last week
Alternatives and similar repositories for llm-d-kv-cache-manager
Users that are interested in llm-d-kv-cache-manager are comparing it to the libraries listed below
Sorting:
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆30Updated 5 months ago
- ☆38Updated 2 weeks ago
- Inference scheduler for llm-d☆41Updated this week
- 💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…☆23Updated 5 months ago
- A distributed engine for intelligent workload☆26Updated 3 months ago
- Resource Exporter for volcano scheduling, e.g. NUMA-Aware scheduling.☆17Updated 7 months ago
- IPAM plugin for kubernetes☆21Updated last week
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆67Updated 3 weeks ago
- ☆37Updated this week
- GenAI inference performance benchmarking tool☆44Updated this week
- Command-line tools for managing OCI model artifacts, which are bundled based on Model Spec☆24Updated this week
- CLI tool that provides helper commands to install, inspect, and troubleshoot a Submariner deployment.☆27Updated this week
- A collection of useful Go libraries to ease the development of NVIDIA Operators for GPU/NIC management.☆24Updated last week
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated 2 years ago
- The Volcano Descheduler☆14Updated 4 months ago
- d.run website☆16Updated last week
- Simplified model deployment on llm-d☆18Updated this week
- The xline-operator is a powerful tool designed to automate the process of bootstrapping, monitoring, snapshotting, and recovering an xlin…☆17Updated last year
- Container Object Storage Interface (COSI) provisioner responsible to interface with COSI drivers. NOTE: The content of this repo has bee…☆34Updated 6 months ago
- Example DRA driver that developers can fork and modify to get them started writing their own.☆73Updated last week
- Canonical location of the Dragonfly API definition☆10Updated this week
- An OCM addon that automates the installation of Kubernetes' konnectivity servers and agents.☆48Updated last month
- Behaviour driven reconciler testing framework.☆30Updated last week
- Operator for managing Node Feature Discovery deployment☆70Updated 2 months ago
- ☆16Updated 3 weeks ago
- Batch-scheduler based on K8s scheduling framework, related features have contributed to scheduler-plugins(Deprecated).☆25Updated 4 years ago
- CNI DRA Driver☆25Updated 4 months ago
- ☆22Updated 3 weeks ago
- ☆12Updated 2 years ago
- Resource Topology exporter for Topology Aware Scheduler☆15Updated last week