fmperf-project / fmperfLinks
Cloud Native Benchmarking of Foundation Models
☆39Updated last week
Alternatives and similar repositories for fmperf
Users that are interested in fmperf are comparing it to the libraries listed below
Sorting:
- A tool to detect infrastructure issues on cloud native AI systems☆44Updated last week
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆202Updated this week
- Systematic and comprehensive benchmarks for LLM systems.☆24Updated last month
- Distributed KV cache coordinator☆46Updated this week
- NVIDIA NCCL Tests for Distributed Training☆102Updated 2 weeks ago
- A light weight vLLM simulator, for mocking out replicas.☆31Updated this week
- CUDA checkpoint and restore utility☆357Updated 6 months ago
- A toolkit for discovering cluster network topology.☆61Updated this week
- GenAI inference performance benchmarking tool☆71Updated this week
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆120Updated this week
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆125Updated last year
- llm-d benchmark scripts and tooling☆21Updated this week
- Holistic job manager on Kubernetes☆117Updated last year
- Inference scheduler for llm-d☆72Updated this week
- Health checks for Azure N- and H-series VMs.☆48Updated this week
- ☆58Updated 10 months ago
- The driver for LMCache core to run in vLLM☆45Updated 6 months ago
- Lightweight daemon for monitoring CUDA runtime API calls with eBPF uprobes☆121Updated 4 months ago
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆196Updated last week
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆69Updated 2 weeks ago
- NVIDIA Inference Xfer Library (NIXL)☆502Updated this week
- Predict the performance of LLM inference services☆19Updated 3 months ago
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆31Updated this week
- GPU scheduler for elastic/distributed deep learning workloads in Kubernetes cluster (IC2E'23)☆35Updated last year
- Microsoft Collective Communication Library☆65Updated 8 months ago
- LLM Serving Performance Evaluation Harness☆79Updated 5 months ago
- An efficient GPU resource sharing system with fine-grained control for Linux platforms.☆84Updated last year
- Stateful LLM Serving☆79Updated 4 months ago
- Helm charts for llm-d☆51Updated 2 weeks ago
- Efficient and easy multi-instance LLM serving☆458Updated this week