Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• ROCm / aiter
  AI Tensor Engine for ROCm
  ☆351Updated this week
• ROCm / TransformerEngine
  ☆60Updated last week
• ROCm / triton
  Development repository for the Triton language and compiler
  ☆140Updated last week
• ROCm / flash-attention
  Fast and memory-efficient exact attention
  ☆214Updated this week
• intel / intel-extension-for-deepspeed
  Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…
  ☆64Updated 7 months ago
• intel / intel-xpu-backend-for-triton
  OpenAI Triton backend for Intel® GPUs
  ☆226Updated this week
• ROCm / rccl-tests
  [DEPRECATED] Moved to ROCm/rocm-systems repo
  ☆86Updated 2 weeks ago
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆111Updated last week
• ROCm / aotriton
  Ahead of Time (AOT) Triton Math Library
  ☆88Updated last week
• mk1-project / quickreduce
  QuickReduce is a performant all-reduce library designed for AMD ROCm that supports inline compression.
  ☆36Updated 5 months ago
• flagos-ai / FlagCX
  FlagCX is a scalable and adaptive cross-chip communication library.
  ☆172Updated this week
• bytedance / xpu-perf
  AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…
  ☆298Updated 3 weeks ago
• ROCm / Megatron-LM
  Ongoing research training transformer models at scale
  ☆36Updated this week
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆85Updated this week
• ROCm / apex
  A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
  ☆25Updated this week
• mlc-ai / relax
  ☆172Updated last week
• ROCm / MAD
  MAD (Model Automation and Dashboarding)
  ☆31Updated this week
• ROCm / hipBLASLt
  [DEPRECATED] Moved to ROCm/rocm-libraries repo
  ☆113Updated this week
• usyd-fsalab / fp6_llm
  An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).
  ☆276Updated 6 months ago
• ROCm / iris
  AMD RAD's multi-GPU Triton-based framework for seamless multi-GPU programming
  ☆168Updated this week
• triton-lang / triton-cpu
  An experimental CPU backend for Triton
  ☆174Updated 3 months ago
• wangsiping97 / FastGEMV
  High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.
  ☆127Updated last year
• BBuf / tensorrt-llm-moe
  ☆34Updated last year
• NVIDIA / nvbandwidth
  A tool for bandwidth measurements on NVIDIA GPUs.
  ☆618Updated 9 months ago
• ROCm / composable_kernel
  [DEPRECATED] Moved to ROCm/rocm-libraries repo. NOTE: develop branch is maintained as a read-only mirror
  ☆518Updated this week
• infinigence / FlashOverlap
  A lightweight design for computation-communication overlap.
  ☆219Updated 2 weeks ago
• ROCm / rocSHMEM
  [DEPRECATED] Moved to ROCm/rocm-systems repo
  ☆144Updated last week
• amd / ZenDNN
  ☆137Updated this week
• DeepLink-org / DLSlime
  DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit
  ☆92Updated 2 weeks ago
• triton-lang / kernels
  ☆104Updated last year