Alternatives and similar repositories for vllm:

Users that are interested in vllm are comparing it to the libraries listed below

• i-am-bee / beeai-code-interpreter
  An HTTP service intended as a backend for an LLM that can run arbitrary pieces of Python code.
  ☆56Updated 2 weeks ago
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆88Updated 11 months ago
• i-am-bee / bee-observe
  Observability API server for bee-agent-framework
  ☆13Updated 3 weeks ago
• JoshuaPurtell / SmallBench
  Small, simple agent task environments for training and evaluation
  ☆18Updated 5 months ago
• cognitivecomputations / kraken
  ☆66Updated 11 months ago
• Just-Curieous / Curie
  ❓Curie: Automated and Rigorous Scientific Experimentation with AI Agents
  ☆77Updated this week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆60Updated 4 months ago
• snowflakedb / ArcticTraining
  ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)
  ☆66Updated this week
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆86Updated last month
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆178Updated this week
• catena-labs / moa-llm
  A Python library to orchestrate LLMs in a neural network-inspired structure
  ☆46Updated 6 months ago
• Xalp / ECHO
  Official homepage for "Self-Harmonized Chain of Thought" (NAACL 2025)
  ☆90Updated 3 months ago
• cognitivecomputations / SystemChat
  ☆30Updated 9 months ago
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆277Updated this week
• facebookresearch / collaborative-reasoner
  Source code for the collaborative reasoner research project at Meta FAIR.
  ☆33Updated last week
• OpenPipe / deductive-reasoning
  Train your own SOTA deductive reasoning model
  ☆88Updated last month
• AI-Hypercomputer / jetstream-pytorch
  PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"
  ☆60Updated 3 weeks ago
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆108Updated 3 months ago
• aniketmaurya / fastserve-ai
  Machine Learning Serving focused on GenAI with simplicity as the top priority.
  ☆58Updated 2 weeks ago
• weaviate / structured-rag
  Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models
  ☆105Updated 2 weeks ago
• huggingface / inference-benchmarker
  Inference server benchmarking tool
  ☆53Updated 3 weeks ago
• JoshuaPurtell / LRCBench
  Evals meant to evaluate language models' ability to reason over long contexts.
  ☆9Updated 7 months ago
• ScalingIntelligence / codemonkeys
  ☆37Updated 2 months ago
• microsoft / agdebugger
  ☆38Updated 2 weeks ago
• axolotl-ai-cloud / grpo_code
  A fast, local, and secure approach for training LLMs for coding tasks using GRPO with WebAssembly and interpreter feedback.
  ☆22Updated 3 weeks ago
• SciPhi-AI / R2R-rag-prebuilt
  ☆11Updated 7 months ago
• guidance-ai / jsonschemabench
  ☆29Updated last week
• cognitivecomputations / grokadamw
  ☆129Updated 8 months ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆262Updated 6 months ago
• langchain-ai / lcel-teacher
  Code-Langchain
  ☆39Updated last year