Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 7 months ago
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆278Updated last week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆61Updated last month
• opea-project / GenAIStudio
  GenAI Studio is a low code platform to enable users to construct, evaluate, and benchmark GenAI applications. The platform also provide c…
  ☆50Updated last month
• apple / ml-recurrent-drafter
  ☆218Updated 8 months ago
• tensorwavecloud / ScalarLM
  ScalarLM - a unified training and inference stack
  ☆85Updated 2 weeks ago
• bentoml / llm-optimizer
  Benchmark and optimize LLM inference across frameworks with ease
  ☆121Updated last month
• cray-lm / cray-lm
  Cray-LM unified training and inference stack.
  ☆22Updated 8 months ago
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆60Updated this week
• cgftinc / benchmax
  Framework-Agnostic RL Environments for LLM Fine-Tuning
  ☆37Updated this week
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆151Updated last week
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆43Updated this week
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆36Updated last week
• brendanhogan / picoDeepResearch
  ☆68Updated 4 months ago
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆55Updated 8 months ago
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆252Updated this week
• substratusai / vllm-docker
  ☆64Updated 6 months ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆266Updated last year
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆83Updated 2 weeks ago
• run-ai / runai-model-streamer
  ☆257Updated this week
• Columbia-NLP-Lab / PAPILLON
  Code for our paper PAPILLON: PrivAcy Preservation from Internet-based and Local Language MOdel ENsembles
  ☆58Updated 5 months ago
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆109Updated 3 weeks ago
• google-deepmind / asyncdiloco
  ☆46Updated last year
• IST-DASLab / gptq-gguf-toolkit
  GPTQ and efficient search for GGUF
  ☆51Updated last month
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆134Updated 4 months ago
• BBischof / yapping
  Verbosity control for AI agents
  ☆65Updated last year
• hetailang / SqueezeAttention
  ☆38Updated last year
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆131Updated 3 weeks ago
• huggingface / tgi-gaudi
  Large Language Model Text Generation Inference on Habana Gaudi
  ☆34Updated 7 months ago
• google-deepmind / latent-multi-hop-reasoning
  [ACL 2024] Do Large Language Models Latently Perform Multi-Hop Reasoning?
  ☆77Updated 7 months ago