Alternatives and similar repositories for vllm:

Users that are interested in vllm are comparing it to the libraries listed below

• ROCm / triton
  Development repository for the Triton language and compiler
  ☆104Updated this week
• ROCm / rocmProfileData
  ☆19Updated this week
• ROCm / pytorch-micro-benchmarking
  ☆18Updated 2 months ago
• usyd-fsalab / fp6_llm
  An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).
  ☆232Updated 3 months ago
• wangsiping97 / FastGEMV
  High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.
  ☆94Updated 6 months ago
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆50Updated this week
• triton-lang / kernels
  ☆64Updated 2 months ago
• ColfaxResearch / cutlass-kernels
  ☆180Updated 6 months ago
• ROCm / Megatron-LM
  Ongoing research training transformer models at scale
  ☆13Updated this week
• ROCm / flash-attention
  Fast and memory-efficient exact attention
  ☆152Updated this week
• ROCm / TransformerEngine
  ☆15Updated this week
• microsoft / vattention
  Dynamic Memory Management for Serving LLMs without PagedAttention
  ☆274Updated this week
• intel / intel-xpu-backend-for-triton
  OpenAI Triton backend for Intel® GPUs
  ☆157Updated this week
• ColfaxResearch / cfx-article-src
  ☆67Updated last month
• microsoft / mscclpp
  MSCCL++: A GPU-driven communication stack for scalable AI applications
  ☆292Updated this week
• bytedance / ByteMLPerf
  AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…
  ☆219Updated 2 weeks ago
• AlibabaPAI / FLASHNN
  ☆79Updated 4 months ago
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆216Updated last week
• intel / torch-xpu-ops
  ☆34Updated this week
• abhibambhaniya / GenZ-LLM-Analyzer
  LLM Inference analyzer for different hardware platforms
  ☆47Updated this week
• ROCm / composable_kernel
  Composable Kernel: Performance Portable Programming Model for Machine Learning Tensor Operators
  ☆338Updated this week
• HandH1998 / QQQ
  QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.
  ☆95Updated last month
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆301Updated 4 months ago
• sunlex0717 / DissectingTensorCores
  ☆84Updated 9 months ago
• Bruce-Lee-LY / cuda_hgemv
  Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.
  ☆54Updated 4 months ago
• OpenPPL / ppl.nn.llm
  ☆140Updated 9 months ago
• ROCm / rccl-tests
  RCCL Performance Benchmark Tests
  ☆55Updated 2 weeks ago
• mit-han-lab / qserve
  QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
  ☆492Updated this week
• ROCm / hipBLASLt
  hipBLASLt is a library that provides general matrix-matrix operations with a flexible API and extends functionalities beyond a traditiona…
  ☆73Updated this week
• RRZE-HPC / gpu-benches
  collection of benchmarks to measure basic GPU capabilities
  ☆287Updated 3 weeks ago