AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.
☆162Mar 6, 2026Updated this week
Alternatives and similar repositories for aiperf
Users that are interested in aiperf are comparing it to the libraries listed below
Sorting:
- ☆18Jun 18, 2025Updated 8 months ago
- Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and i…☆31Feb 27, 2026Updated last week
- Distributed KV cache scheduling & offloading libraries☆108Updated this week
- Example of applying CUDA graphs to LLaMA-v2☆12Aug 25, 2023Updated 2 years ago
- llm-d helm charts and deployment examples☆50Feb 26, 2026Updated last week
- Prometheus exporter for lustre☆23Updated this week
- Test Orchestrator for Performance and Scalability of AI pLatforms☆16Feb 27, 2026Updated last week
- A light weight vLLM simulator, for mocking out replicas.☆87Updated this week
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆26Apr 24, 2025Updated 10 months ago
- A workload for deploying LLM inference services on Kubernetes☆179Updated this week
- The Intelligent Inference Scheduler for Large-scale Inference Services.☆64Feb 12, 2026Updated 3 weeks ago
- DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit☆92Jan 26, 2026Updated last month
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆275Updated this week
- High Performance KV Cache Store for LLM☆47Updated this week
- caniuse.com, but for kubernetes☆27Dec 25, 2024Updated last year
- A high-performance and light-weight router for vLLM large scale deployment☆131Updated this week
- A collection of useful Go libraries to ease the development of NVIDIA Operators for GPU/NIC management.☆29Updated this week
- ☆24Jun 24, 2022Updated 3 years ago
- Stable-DiffCoder is a family of lightweight open-source code DLLMs(diffusion large language models) comprising base and instruct models, …☆75Jan 23, 2026Updated last month
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆74Jul 18, 2025Updated 7 months ago
- ☆34Feb 3, 2025Updated last year
- mHC-lite: You Don’t Need 20 Sinkhorn-Knopp Iterations☆70Jan 12, 2026Updated last month
- The code used to evaluate embedding models on the Massive Legal Embedding Benchmark (MLEB).☆31Feb 24, 2026Updated last week
- Multi-step AI agents powered by Gemini 2.0 and the LangGraph framework. These agents orchestrate complex workflows and enhance their reas…☆10Dec 19, 2024Updated last year
- Advanced block device testing/file system testing, targetting SNIA compatible reporting☆12Oct 15, 2025Updated 4 months ago
- [WIP] Better (FP8) attention for Hopper☆32Feb 24, 2025Updated last year
- NVIDIA's launch, startup, and logging scripts used by our MLPerf Training and HPC submissions☆35Sep 12, 2025Updated 5 months ago
- A distributed system for Agentic AI☆47Updated this week
- KV cache store for distributed LLM inference☆396Nov 13, 2025Updated 3 months ago
- Systematic and comprehensive benchmarks for LLM systems.☆51Jan 28, 2026Updated last month
- A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.☆30Mar 28, 2025Updated 11 months ago
- Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, T…☆384Updated this week
- AI-native knowledge kernel for human/agent collaboration. Use it as a Knowledge Base, Wiki, Annotator, Research Tool, or Agentic Memory.☆29Updated this week
- Lustre Repository with MS patches☆13Feb 28, 2026Updated last week
- ☆10Aug 7, 2024Updated last year
- Python wrappers for the FirecREST API☆12Dec 23, 2025Updated 2 months ago
- Smart Kubernetes Scheduling☆83Feb 27, 2026Updated last week
- Martingale posterior neural networks for fast sequential decision making @ Neurips 2025☆23Nov 13, 2025Updated 3 months ago
- Software to enable data-rich collaboration from high-resolution display walls to your laptop☆16Updated this week