Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• InferenceMAX / InferenceMAX
  Open Source Continuous Inference Benchmarking - GB200 NVL72 vs MI355X vs B200 vs H200 vs MI325X & soon™ TPUv6e/v7/Trainium2/3/GB300 NVL72…
  ☆405Updated this week
• NVIDIA-NeMo / Nemotron
  Developer Asset Hub for NVIDIA Nemotron — A one-stop resource for training recipes, usage cookbooks, and full end-to-end reference exampl…
  ☆246Updated last week
• bentoml / llm-optimizer
  Benchmark and optimize LLM inference across frameworks with ease
  ☆151Updated 3 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆93Updated 9 months ago
• opea-project / GenAIStudio
  GenAI Studio is a low code platform to enable users to construct, evaluate, and benchmark GenAI applications. The platform also provide c…
  ☆55Updated 2 weeks ago
• tensorwavecloud / ScalarLM
  ScalarLM - a unified training and inference stack
  ☆93Updated last month
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆273Updated this week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆62Updated 3 months ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆163Updated last month
• cray-lm / cray-lm
  Cray-LM unified training and inference stack.
  ☆22Updated 11 months ago
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆147Updated last week
• tilde-research / MoMoE-impl
  Memory optimized Mixture of Experts
  ☆72Updated 5 months ago
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆59Updated 2 months ago
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆357Updated this week
• kurakurai / Luth
  Luth is a state-of-the-art series of fine-tuned LLMs for French
  ☆41Updated 2 months ago
• facebookresearch / matrix
  Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…
  ☆246Updated last week
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆44Updated last week
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆137Updated 6 months ago
• run-ai / runai-model-streamer
  ☆273Updated last week
• VectorInstitute / fed-rag
  A framework for fine-tuning retrieval-augmented generation (RAG) systems.
  ☆137Updated last week
• brendanhogan / picoDeepResearch
  ☆68Updated 7 months ago
• nexusflowai / NexusBench
  Nexusflow function call, tool use, and agent benchmarks.
  ☆30Updated last year
• google-deepmind / latent-multi-hop-reasoning
  [ACL 2024] Do Large Language Models Latently Perform Multi-Hop Reasoning?
  ☆84Updated 9 months ago
• cgftinc / benchmax
  Framework-Agnostic RL Environments for LLM Fine-Tuning
  ☆40Updated last month
• axolotl-ai-cloud / axolotl-cookbook
  ☆36Updated 4 months ago
• substratusai / vllm-docker
  ☆67Updated 9 months ago
• OpenPipe / open_deep_research_training
  Training setup for Langchain's Open Deep Research
  ☆74Updated 4 months ago
• raphaelmansuy / iteration_of_tought
  Example implementation of Iteration of Tought - Gives a star if you like the project
  ☆41Updated last year
• dropbox / aana_sdk
  Aana SDK is a powerful framework for building AI enabled multimodal applications.
  ☆55Updated 4 months ago
• snowflakedb / ArcticTraining
  ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)
  ☆263Updated this week