Alternatives and similar repositories for genai-bench

Users that are interested in genai-bench are comparing it to the libraries listed below

• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆407Updated 3 months ago
• fzyzcjy / torch_memory_saver
  Allow torch tensor memory to be released and resumed later
  ☆115Updated 2 weeks ago
• perplexityai / pplx-kernels
  Perplexity GPU Kernels
  ☆449Updated 3 weeks ago
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆408Updated 3 months ago
• anyscale / llm-continuous-batching-benchmarks
  ☆121Updated last year
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆79Updated 6 months ago
• sgl-project / SpecForge
  Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
  ☆325Updated last week
• stepfun-ai / StepMesh
  ☆284Updated this week
• InternLM / turbomind
  ☆92Updated 5 months ago
• sgl-project / sgl-learning-materials
  Materials for learning SGLang
  ☆549Updated last week
• WukLab / preble
  Stateful LLM Serving
  ☆81Updated 5 months ago
• sail-sg / zero-bubble-pipeline-parallelism
  Zero Bubble Pipeline Parallelism
  ☆421Updated 3 months ago
• ByteDance-Seed / ByteCheckpoint
  ByteCheckpoint: An Unified Checkpointing Library for LFMs
  ☆238Updated last month
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆473Updated last week
• cli99 / llm-analysis
  Latency and Memory Analysis of Transformer Models for Training and Inference
  ☆449Updated 4 months ago
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆223Updated this week
• MDK8888 / vllmini
  A minimal implementation of vllm.
  ☆51Updated last year
• RulinShao / LightSeq
  Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training
  ☆214Updated last year
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆180Updated 11 months ago
• efeslab / Atom
  [MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving
  ☆319Updated last year
• AlibabaPAI / torchacc
  PyTorch distributed training acceleration framework
  ☆52Updated 2 weeks ago
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆91Updated this week
• hao-ai-lab / MuxServe
  ☆69Updated last year
• meta-pytorch / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆292Updated last week
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆128Updated last year
• microsoft / nnscaler
  nnScaler: Compiling DNN models for Parallel Training
  ☆115Updated last week
• neuralmagic / AutoFP8
  ☆196Updated 3 months ago
• vllm-project / recipes
  Common recipes to run vLLM
  ☆110Updated last week
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆47Updated 6 months ago
• EfficientMoE / MoE-Infinity
  PyTorch library for cost-effective, fast and easy serving of MoE models.
  ☆224Updated last month