kubernetes-sigs / inference-perf
GenAI inference performance benchmarking tool
☆41Updated this week
Alternatives and similar repositories for inference-perf:
Users that are interested in inference-perf are comparing it to the libraries listed below
- Example DRA driver that developers can fork and modify to get them started writing their own.☆69Updated last month
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆97Updated this week
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆66Updated this week
- Cloud Native Artifacial Intelligence Model Format Specification☆41Updated last week
- Gateway API Inference Extension☆268Updated this week
- JobSet: a k8s native API for distributed ML training and HPC workloads☆223Updated this week
- WG Serving☆24Updated 2 weeks ago
- Kubernetes Work API☆66Updated last month
- Libraries for implementing aggregated apiservers☆89Updated 2 weeks ago
- Holistic job manager on Kubernetes☆115Updated last year
- ☆148Updated 2 weeks ago
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated last year
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆29Updated 4 months ago
- InstaSlice Operator facilitates slicing of accelerators using stable APIs☆34Updated this week
- InstaSlice facilitates the use of Dynamic Resource Allocation (DRA) on Kubernetes clusters for GPU sharing☆27Updated 5 months ago
- ☆16Updated this week
- ☆114Updated 2 years ago
- CAPK is a provider for Cluster API (CAPI) that allows users to deploy fake, Kubemark-backed machines to their clusters.☆72Updated last month
- Command-line tools for managing OCI model artifacts, which are bundled based on Model Spec☆20Updated last week
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆160Updated this week
- Operator for managing Node Feature Discovery deployment☆69Updated last month
- AppWrapper controller for Kueue☆12Updated last week
- ☆51Updated last year
- The kernel module management operator builds, signs and loads kernel modules in Kubernetes clusters.☆101Updated last week
- Following the same workflows as Kubernetes. Widely used in InftyAI community.☆12Updated last week
- ☆110Updated this week
- CNI DRA Driver☆25Updated 3 months ago
- Addon operators for Kubernetes clusters.☆157Updated 2 years ago
- Idle containers when not handling requests.☆41Updated 2 years ago
- A collection of community maintained NRI plugins☆79Updated 2 weeks ago