Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• sgl-project / sglang
  SGLang is a fast serving framework for large language models and vision language models.
  ☆18,662Updated this week
• unslothai / unsloth
  Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train OpenAI gpt-oss, DeepSeek-R1, Qwen3, Gemma 3, TTS 2x faster with 70% less VRAM.
  ☆46,777Updated this week
• huggingface / peft
  🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
  ☆19,781Updated this week
• InternLM / lmdeploy
  LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
  ☆7,146Updated this week
• huggingface / text-generation-inference
  Large Language Model Text Generation Inference
  ☆10,550Updated 3 weeks ago
• lm-sys / FastChat
  An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
  ☆39,141Updated 4 months ago
• NVIDIA / TensorRT-LLM
  TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizati…
  ☆11,801Updated this week
• Dao-AILab / flash-attention
  Fast and memory-efficient exact attention
  ☆19,864Updated this week
• artidoro / qlora
  QLoRA: Efficient Finetuning of Quantized LLMs
  ☆10,680Updated last year
• mlc-ai / mlc-llm
  Universal LLM Deployment Engine with ML Compilation
  ☆21,440Updated last week
• run-llama / llama_index
  LlamaIndex is the leading framework for building LLM-powered agents over your data.
  ☆44,665Updated this week
• meta-llama / llama-cookbook
  Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We als…
  ☆17,938Updated this week
• meta-llama / llama
  Inference code for Llama models
  ☆58,807Updated 8 months ago
• ggml-org / llama.cpp
  LLM inference in C/C++
  ☆87,385Updated this week
• bitsandbytes-foundation / bitsandbytes
  Accessible large language models via k-bit quantization for PyTorch.
  ☆7,647Updated last week
• huggingface / trl
  Train transformer language models with reinforcement learning.
  ☆15,818Updated this week
• BerriAI / litellm
  Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sag…
  ☆29,802Updated this week
• abetlen / llama-cpp-python
  Python bindings for llama.cpp
  ☆9,647Updated last month
• chroma-core / chroma
  Open-source search and retrieval database for AI applications.
  ☆23,781Updated this week
• FlagOpen / FlagEmbedding
  Retrieval and Retrieval-augmented LLMs
  ☆10,655Updated this week
• haotian-liu / LLaVA
  [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
  ☆23,699Updated last year
• BlinkDL / RWKV-LM
  RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable)…
  ☆14,009Updated this week
• ggml-org / ggml
  Tensor library for machine learning
  ☆13,261Updated this week
• NVIDIA / Megatron-LM
  Ongoing research training transformer models at scale
  ☆13,755Updated last week
• axolotl-ai-cloud / axolotl
  Go ahead and axolotl questions
  ☆10,592Updated this week
• oobabooga / text-generation-webui
  The definitive Web UI for local AI, with powerful features and easy setup.
  ☆45,135Updated this week
• EleutherAI / lm-evaluation-harness
  A framework for few-shot evaluation of language models.
  ☆10,303Updated this week
• ollama / ollama
  Get up and running with OpenAI gpt-oss, DeepSeek-R1, Gemma 3 and other models.
  ☆153,970Updated this week
• gradio-app / gradio
  Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!
  ☆40,169Updated this week
• langchain-ai / langchain
  🦜🔗 Build context-aware reasoning applications
  ☆116,801Updated this week