Cloud Native Benchmarking of Foundation Models
☆46Jul 31, 2025Updated last year
Alternatives and similar repositories for fmperf
Users that are interested in fmperf are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Predict the performance of LLM inference services☆23Sep 18, 2025Updated last year
- ☆21Oct 24, 2026Updated last week
- Community maintained hardware plugin for vLLM on Spyre☆53Oct 1, 2026Updated last month
- A tool to detect infrastructure issues on cloud native AI systems☆56Sep 18, 2025Updated last year
- Test Orchestrator for Performance and Scalability of AI pLatforms☆18Jun 23, 2026Updated 4 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Simplified model deployment on llm-d☆28Jul 2, 2025Updated last year
- ☆67Updated this week
- Model as a Service☆46Updated this week
- Self-hosted, distributed, multi-model LLM inference☆42Updated this week
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆35Updated this week
- Systematic and comprehensive benchmarks for LLM systems.☆61Jan 28, 2026Updated 9 months ago
- llm-d-batch-gateway is a standalone, backend-agnostic, OpenAI-compatible batch API and processing engine☆28Updated this week
- ⚠️ ARCHIVED - All development moved to https://github.com/itbench-hub/ITBench/tree/main/scenarios☆16Feb 24, 2026Updated 8 months ago
- Helm charts for llm-d☆52Jul 22, 2025Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆345Updated this week
- How to create and record demos in terminal sessions☆11May 3, 2024Updated 2 years ago
- GenAI inference performance benchmarking tool☆267Oct 28, 2026Updated last week
- A Kubernetes Operator to manage Node OS customizations.☆75Updated this week
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆27Apr 24, 2025Updated last year
- Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and i…☆198Updated this week
- ☆14May 11, 2026Updated 5 months ago
- ☆22Mar 11, 2026Updated 7 months ago
- KV cache store for distributed LLM inference☆451Sep 16, 2026Updated last month
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆139Feb 22, 2024Updated 2 years ago
- NVSentinel detects and remediates GPU faults on Kubernetes nodes☆423Updated this week
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆228Oct 14, 2026Updated 2 weeks ago
- Carbon Limiting Auto Tuning for Kubernetes☆38Mar 19, 2026Updated 7 months ago
- ☆17May 8, 2020Updated 6 years ago
- Incubating P/D sidecar for llm-d☆17Nov 13, 2025Updated 11 months ago
- Sardeenz is a proof-of-concept application that allows you to load more than one model on a given GPU. It allows you to add more and more…☆65Aug 14, 2026Updated 2 months ago
- llm-d benchmark scripts and tooling☆77Updated this week
- Prism: Revealing Hidden Functional Clusters from Massive Instances in Cloud Systems [ASE'2023]☆10Jan 10, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Experimental Linear Algebra Performance Studies☆12Feb 24, 2017Updated 9 years ago
- ☆126Nov 1, 2022Updated 4 years ago
- Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, T…☆523Updated this week
- This is the implementation repository of our OSDI'23 paper: SMART: A High-Performance Adaptive Radix Tree for Disaggregated Memory.☆64Oct 28, 2024Updated 2 years ago
- ☆21Sep 9, 2024Updated 2 years ago
- Measure how long it takes for a program to execute in different clocks☆12Nov 1, 2021Updated 5 years ago
- Apache OpenWhisk Composer provides a high-level programming model in JavaScript for composing serverless functions☆68Sep 24, 2024Updated 2 years ago