Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 6 months ago
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆42Updated this week
• facebookresearch / matrix
  Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…
  ☆88Updated this week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆61Updated 3 months ago
• substratusai / vllm-docker
  ☆63Updated 5 months ago
• cray-lm / cray-lm
  Cray-LM unified training and inference stack.
  ☆22Updated 7 months ago
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆223Updated last week
• tensorwavecloud / ScalarLM
  ScalarLM - a unified training and inference stack
  ☆55Updated 3 weeks ago
• parea-ai / parea-sdk-py
  Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
  ☆78Updated 6 months ago
• opea-project / GenAIStudio
  GenAI Studio is a low code platform to enable users to construct, evaluate, and benchmark GenAI applications. The platform also provide c…
  ☆48Updated last week
• brendanhogan / picoDeepResearch
  ☆68Updated 3 months ago
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆223Updated this week
• apple / ml-recurrent-drafter
  ☆217Updated 7 months ago
• google-deepmind / asyncdiloco
  ☆46Updated last year
• BBischof / yapping
  Verbosity control for AI agents
  ☆65Updated last year
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆97Updated 2 months ago
• axolotl-ai-cloud / axolotl-cookbook
  ☆34Updated last month
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆267Updated 10 months ago
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆36Updated last month
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆131Updated 2 months ago
• aniketmaurya / fastserve-ai
  Machine Learning Serving focused on GenAI with simplicity as the top priority.
  ☆59Updated last month
• OpenPipe / deductive-reasoning
  Train your own SOTA deductive reasoning model
  ☆104Updated 5 months ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆140Updated this week
• google-deepmind / latent-multi-hop-reasoning
  [ACL 2024] Do Large Language Models Latently Perform Multi-Hop Reasoning?
  ☆74Updated 5 months ago
• unslothai / unsloth-zoo
  Utils for Unsloth https://github.com/unslothai/unsloth
  ☆134Updated this week
• Columbia-NLP-Lab / PAPILLON
  Code for our paper PAPILLON: PrivAcy Preservation from Internet-based and Local Language MOdel ENsembles
  ☆54Updated 3 months ago
• guidance-ai / jsonschemabench
  ☆55Updated 2 months ago
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆55Updated 7 months ago
• run-ai / runai-model-streamer
  ☆238Updated this week
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆120Updated last year