Alternatives and similar repositories for aiperf

Users that are interested in aiperf are comparing it to the libraries listed below

• coreweave / nccl-tests
  NVIDIA NCCL Tests for Distributed Training
  ☆123Updated last week
• ai-dynamo / aiconfigurator
  Offline optimization of your disaggregated Dynamo graph
  ☆106Updated this week
• ai-dynamo / nixl
  NVIDIA Inference Xfer Library (NIXL)
  ☆721Updated this week
• NVIDIA / nvidia-resiliency-ext
  NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …
  ☆234Updated this week
• sgl-project / ome
  OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)
  ☆312Updated this week
• triton-inference-server / perf_analyzer
  ☆118Updated last week
• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆230Updated this week
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆445Updated last month
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆85Updated last week
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆510Updated 2 months ago
• google / nccl-fastsocket
  NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.
  ☆122Updated 2 years ago
• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆436Updated 5 months ago
• LMCache / LMBenchmark
  Systematic and comprehensive benchmarks for LLM systems.
  ☆40Updated last month
• microsoft / NPKit
  NCCL Profiling Kit
  ☆147Updated last year
• perplexityai / pplx-kernels
  Perplexity GPU Kernels
  ☆529Updated 2 weeks ago
• facebookresearch / param
  PArametrized Recommendation and Ai Model benchmark is a repository for development of numerous uBenchmarks as well as end to end nets for…
  ☆153Updated this week
• NVIDIA / nvbandwidth
  A tool for bandwidth measurements on NVIDIA GPUs.
  ☆568Updated 7 months ago
• mk1-project / quickreduce
  QuickReduce is a performant all-reduce library designed for AMD ROCm that supports inline compression.
  ☆35Updated 2 months ago
• fmperf-project / fmperf
  Cloud Native Benchmarking of Foundation Models
  ☆44Updated 3 months ago
• Azure / msccl
  Microsoft Collective Communication Library
  ☆66Updated 11 months ago
• NVIDIA / cuda-checkpoint
  CUDA checkpoint and restore utility
  ☆384Updated 2 months ago
• eth-easl / orion
  An interference-aware scheduler for fine-grained GPU sharing
  ☆152Updated 9 months ago
• ovg-project / kvcached
  Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
  ☆661Updated last week
• Mellanox / nccl-rdma-sharp-plugins
  RDMA and SHARP plugins for nccl library
  ☆212Updated 3 weeks ago
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆132Updated last year
• Sys-KU / DeepPlan
  [ACM EuroSys 2023] Fast and Efficient Model Serving Using Multi-GPUs with Direct-Host-Access
  ☆57Updated 3 months ago
• meta-pytorch / torchcomms
  torchcomms: a modern PyTorch communications API
  ☆278Updated last week
• perplexityai / libfabric-efa-demo
  ☆72Updated 9 months ago
• sakjain92 / Fractional-GPUs
  Splits single Nvidia GPU into multiple partitions with complete compute and memory isolation (wrt to performace) between the partitions
  ☆163Updated 6 years ago
• aws / aws-ofi-nccl
  This is a plugin which lets EC2 developers use libfabric as network provider while running NCCL applications.
  ☆193Updated this week