Alternatives and similar repositories for genai-bench

Users that are interested in genai-bench are comparing it to the libraries listed below

• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆390Updated last month
• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆401Updated last month
• ppl-ai / pplx-kernels
  Perplexity GPU Kernels
  ☆405Updated this week
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆79Updated 4 months ago
• ByteDance-Seed / ByteCheckpoint
  ByteCheckpoint: An Unified Checkpointing Library for LFMs
  ☆226Updated last week
• sail-sg / zero-bubble-pipeline-parallelism
  Zero Bubble Pipeline Parallelism
  ☆406Updated 2 months ago
• fzyzcjy / torch_memory_saver
  Allow torch tensor memory to be released and resumed later
  ☆80Updated last week
• EfficientMoE / MoE-Infinity
  PyTorch library for cost-effective, fast and easy serving of MoE models.
  ☆209Updated last week
• anyscale / llm-continuous-batching-benchmarks
  ☆120Updated last year
• cli99 / llm-analysis
  Latency and Memory Analysis of Transformer Models for Training and Inference
  ☆435Updated 2 months ago
• microsoft / nnscaler
  nnScaler: Compiling DNN models for Parallel Training
  ☆113Updated last week
• InternLM / turbomind
  ☆90Updated 3 months ago
• WukLab / preble
  Stateful LLM Serving
  ☆76Updated 4 months ago
• RulinShao / LightSeq
  Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training
  ☆212Updated 10 months ago
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆168Updated 9 months ago
• alibaba / easydist
  Automated Parallelization System and Infrastructure for Multiple Ecosystems
  ☆79Updated 7 months ago
• hao-ai-lab / MuxServe
  ☆64Updated last year
• infinigence / FlashOverlap
  A lightweight design for computation-communication overlap.
  ☆146Updated 3 weeks ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆121Updated 7 months ago
• tgale96 / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆101Updated last month
• efeslab / Atom
  [MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving
  ☆314Updated last year
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆69Updated 2 months ago
• sgl-project / sgl-learning-materials
  Materials for learning SGLang
  ☆481Updated last week
• Dao-AILab / quack
  A Quirky Assortment of CuTe Kernels
  ☆281Updated this week
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆281Updated last month
• mobiusml / gemlite
  Fast low-bit matmul kernels in Triton
  ☆330Updated last week
• LoongServe / LoongServe
  ☆106Updated 8 months ago
• infinigence / Semi-PD
  A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
  ☆95Updated 2 months ago
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆448Updated this week
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆123Updated last year