Alternatives and similar repositories for aiperf

Users that are interested in aiperf are comparing it to the libraries listed below

• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆263Updated last week
• ai-dynamo / nixl
  NVIDIA Inference Xfer Library (NIXL)
  ☆876Updated this week
• perplexityai / pplx-kernels
  Perplexity GPU Kernels
  ☆560Updated 3 months ago
• meta-pytorch / torchcomms
  torchcomms: a modern PyTorch communications API
  ☆330Updated this week
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆391Updated this week
• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆458Updated 8 months ago
• NVIDIA / nvidia-resiliency-ext
  NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …
  ☆262Updated this week
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆470Updated last month
• sgl-project / ome
  Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, T…
  ☆365Updated last week
• triton-inference-server / perf_analyzer
  ☆134Updated last week
• vllm-project / router
  A high-performance and light-weight router for vLLM large scale deployment
  ☆112Updated this week
• coreweave / nccl-tests
  NVIDIA NCCL Tests for Distributed Training
  ☆134Updated 2 weeks ago
• perplexityai / pplx-garden
  Perplexity open source garden for inference technology
  ☆362Updated last month
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆85Updated last week
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆228Updated this week
• NVIDIA / nvshmem
  NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process com…
  ☆462Updated last month
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆524Updated 5 months ago
• vllm-project / tpu-inference
  TPU inference for vLLM, with unified JAX and PyTorch support.
  ☆228Updated last week
• ovg-project / kvcached
  Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
  ☆773Updated this week
• ai-dynamo / aiconfigurator
  Offline optimization of your disaggregated Dynamo graph
  ☆184Updated this week
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆390Updated 2 months ago
• LMCache / LMBenchmark
  Systematic and comprehensive benchmarks for LLM systems.
  ☆50Updated 2 weeks ago
• Azure / msccl
  Microsoft Collective Communication Library
  ☆66Updated last year
• microsoft / mscclpp
  MSCCL++: A GPU-driven communication stack for scalable AI applications
  ☆462Updated this week
• meta-pytorch / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆315Updated 5 months ago
• run-ai / llmperf
  ☆61Updated last year
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆60Updated last year
• fzyzcjy / torch_memory_saver
  Allow torch tensor memory to be released and resumed later
  ☆216Updated 3 weeks ago
• NVIDIA / cuda-checkpoint
  CUDA checkpoint and restore utility
  ☆410Updated 4 months ago
• mk1-project / quickreduce
  QuickReduce is a performant all-reduce library designed for AMD ROCm that supports inline compression.
  ☆36Updated 5 months ago