kubernetes-sigs / inference-perf
GenAI inference performance benchmarking tool
☆36Updated last week
Alternatives and similar repositories for inference-perf:
Users that are interested in inference-perf are comparing it to the libraries listed below
- Example DRA driver that developers can fork and modify to get them started writing their own.☆69Updated 3 weeks ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆64Updated 2 weeks ago
- Gateway API Inference Extension☆220Updated this week
- JobSet: a k8s native API for distributed ML training and HPC workloads☆217Updated this week
- Operator for managing Node Feature Discovery deployment☆69Updated 2 weeks ago
- Holistic job manager on Kubernetes☆114Updated last year
- WG Serving☆23Updated last month
- Kubernetes Work API☆66Updated 2 weeks ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆92Updated this week
- 🏃🏿♀️🏃🏽♀️🏃🏻♂️🕒CNCF Technical Advisory Group for Runtime☆92Updated 3 months ago
- All the things to make the scheduler extendable with wasm.☆116Updated 2 months ago
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12Updated last year
- Libraries for implementing aggregated apiservers☆88Updated 3 weeks ago
- ☆35Updated this week
- ☆125Updated this week
- ☆21Updated last week
- Cloud Native Artifacial Intelligence Model Format Specification☆40Updated 3 weeks ago
- ☆51Updated last year
- ☆114Updated 2 years ago
- InstaSlice facilitates the use of Dynamic Resource Allocation (DRA) on Kubernetes clusters for GPU sharing☆27Updated 4 months ago
- The official Kubernetes operator for etcd.☆56Updated this week
- Enabling Kubernetes to make pod placement decisions with platform intelligence.☆174Updated 2 months ago
- ☆16Updated 2 weeks ago
- 💫 A lightweight p2p-based cache system for model distributions on Kubernetes. Reframing now to make it an unified cache system with POSI…☆20Updated 4 months ago
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆29Updated 3 months ago
- InstaSlice Operator facilitates slicing of accelerators using stable APIs☆33Updated this week
- ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!☆120Updated this week
- [Moved to https://github.com/kubernetes-sigs/kwok] This is a fake kubelet. that can simulate any number of nodes and maintain pods on tho…☆63Updated 2 years ago
- A simulator of Kuberntes for batch and service workload.☆46Updated 4 years ago
- A collection of community maintained NRI plugins☆78Updated last week