fmperf-project / fmperfLinks
Cloud Native Benchmarking of Foundation Models
☆36Updated 3 weeks ago
Alternatives and similar repositories for fmperf
Users that are interested in fmperf are comparing it to the libraries listed below
Sorting:
- A tool to detect infrastructure issues on cloud native AI systems☆36Updated 2 weeks ago
- ☆15Updated this week
- A light weight vLLM simulator, for mocking out replicas.☆19Updated last week
- NVIDIA NCCL Tests for Distributed Training☆92Updated last week
- GenAI inference performance benchmarking tool☆45Updated this week
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆107Updated this week
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆121Updated last year
- Holistic job manager on Kubernetes☆115Updated last year
- Inference scheduler for llm-d☆47Updated last week
- An interference-aware scheduler for fine-grained GPU sharing☆137Updated 4 months ago
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆30Updated 5 months ago
- ☆49Updated 2 months ago
- GPU scheduler for elastic/distributed deep learning workloads in Kubernetes cluster (IC2E'23)☆34Updated last year
- Distributed KV cache coordinator☆31Updated 2 weeks ago
- Tiresias is a GPU cluster manager for distributed deep learning training.☆154Updated 5 years ago
- knavigator is a development, testing, and optimization toolkit for AI/ML scheduling systems at scale on Kubernetes.☆67Updated last month
- An efficient GPU resource sharing system with fine-grained control for Linux platforms.☆83Updated last year
- Microsoft Collective Communication Library☆65Updated 6 months ago
- Repository to demo GPU Sharing with Time Slicing, MPS, MIG and others☆42Updated 7 months ago
- NCCL Profiling Kit☆135Updated 11 months ago
- Health checks for Azure N- and H-series VMs.☆42Updated last month
- Intelligent platform for AI workloads☆37Updated 2 years ago
- Lightweight daemon for monitoring CUDA runtime API calls with eBPF uprobes☆96Updated 2 months ago
- Predict the performance of LLM inference services☆18Updated last month
- ☆54Updated 8 months ago
- Artifacts for our NSDI'23 paper TGS☆75Updated 11 months ago
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆173Updated this week
- Fine-grained GPU sharing primitives☆141Updated 5 years ago
- Helm charts for llm-d☆35Updated last week
- NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.☆116Updated last year