fmperf-project / fmperfLinks
Cloud Native Benchmarking of Foundation Models
☆44Updated 3 months ago
Alternatives and similar repositories for fmperf
Users that are interested in fmperf are comparing it to the libraries listed below
Sorting:
- A tool to detect infrastructure issues on cloud native AI systems☆49Updated last month
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆298Updated this week
- NVIDIA NCCL Tests for Distributed Training☆116Updated last week
- llm-d benchmark scripts and tooling☆30Updated last week
- Systematic and comprehensive benchmarks for LLM systems.☆38Updated last month
- Distributed KV cache coordinator☆80Updated this week
- Offline optimization of your disaggregated Dynamo graph☆88Updated this week
- CUDA checkpoint and restore utility☆377Updated last month
- A toolkit for discovering cluster network topology.☆74Updated this week
- Health checks for Azure N- and H-series VMs.☆54Updated 3 weeks ago
- A light weight vLLM simulator, for mocking out replicas.☆55Updated this week
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆222Updated last week
- Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond☆469Updated this week
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆130Updated last year
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆228Updated last week
- GenAI inference performance benchmarking tool☆106Updated 2 weeks ago
- ☆58Updated last year
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆130Updated this week
- A workload for deploying LLM inference services on Kubernetes☆87Updated this week
- GPU scheduler for elastic/distributed deep learning workloads in Kubernetes cluster (IC2E'23)☆35Updated last year
- Holistic job manager on Kubernetes☆116Updated last year
- Microsoft Collective Communication Library☆66Updated 11 months ago
- AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solu…☆30Updated this week
- NVIDIA Inference Xfer Library (NIXL)☆688Updated this week
- LLM Serving Performance Evaluation Harness☆79Updated 8 months ago
- MIG Partition Editor for NVIDIA GPUs☆219Updated last week
- Efficient and easy multi-instance LLM serving☆502Updated last month
- An efficient GPU resource sharing system with fine-grained control for Linux platforms.☆85Updated last year
- Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling☆82Updated this week
- Automatic tuning for ML model deployment on Kubernetes☆81Updated last year