Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• LLMServe / SwiftTransformer
  High performance Transformer implementation in C++.
  ☆142Updated 10 months ago
• alibaba-edu / qwen-bailian-usagetraces-anon
  ☆61Updated last month
• ASISys / Adrenaline
  Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation
  ☆38Updated 2 weeks ago
• zartbot / shallowsim
  DeepSeek-V3/R1 inference performance simulator
  ☆168Updated 8 months ago
• shenh10 / DeepSeek_Simulator
  ☆90Updated 7 months ago
• lipracer / cuda-rt-hook
  ☆45Updated 4 months ago
• infinigence / Semi-PD
  A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
  ☆116Updated 6 months ago
• jychen21 / Habana-LLM-Viewer
  ☆11Updated last year
• SJTU-IPADS / reef
  REEF is a GPU-accelerated DNN inference serving system that enables instant kernel preemption and biased concurrent execution in GPU sche…
  ☆103Updated 2 years ago
• DeepLink-org / DLSlime
  DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit
  ☆82Updated this week
• Multi-LLM / prism-research
  Research prototype of PRISM — a cost-efficient multi-LLM serving system with flexible time- and space-based GPU sharing.
  ☆46Updated 3 months ago
• eniac / paella
  Paella: Low-latency Model Serving with Virtualized GPU Scheduling
  ☆65Updated last year
• SJTU-IPADS / ugache
  ☆23Updated 2 years ago
• pkusys / Rummy
  GPU-accelerated vector query processing system that supports large vector datasets beyond GPU memory.
  ☆34Updated last year
• sunkx109 / GPUs-Specs
  Summary of the Specs of Commonly Used GPUs for Training and Inference of LLM
  ☆64Updated 3 months ago
• alibaba / llm-scheduling-artifact
  Artifact of OSDI '24 paper, ”Llumnix: Dynamic Scheduling for Large Language Model Serving“
  ☆63Updated last year
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆510Updated 2 months ago
• SJTU-IPADS / PhoenixOS
  Fast OS-level support for GPU checkpoint and restore
  ☆257Updated 2 months ago
• Thesys-lab / Helix-ASPLOS25
  Open-source implementation for "Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow"
  ☆73Updated last month
• WukLab / preble
  Stateful LLM Serving
  ☆89Updated 8 months ago
• LLMServe / dLoRA-artifact
  ☆27Updated last year
• thustorage / Medusa
  Medusa: Accelerating Serverless LLM Inference with Materialization [ASPLOS'25]
  ☆39Updated 6 months ago
• YaoJiayi / CacheBlend
  ☆152Updated 4 months ago
• tile-ai / tilelang-ascend
  Ascend TileLang adapter
  ☆149Updated this week
• Hsword / Awesome-Machine-Learning-System-Papers
  ☆79Updated 3 years ago
• infinigence / SpecEE
  Repo for SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting (ISCA25)
  ☆68Updated 7 months ago
• DicardoX / Research-Space
  This repository is established to store personal notes and annotated papers during daily research.
  ☆162Updated last week
• PDZZXL / Awesome-LLM-Serving
  Large Language Model (LLM) Serving Paper and Resource List
  ☆24Updated 6 months ago
• nono-Sang / light-rpc
  An Efficient RDMA-based RPC Framework
  ☆23Updated 2 years ago
• interestingLSY / CUDA-From-Correctness-To-Performance-Code
  Codes & examples for "CUDA - From Correctness to Performance"
  ☆117Updated last year