Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• cray-lm / cray-lm
  Cray-LM unified training and inference stack.
  ☆22Updated 9 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆92Updated 8 months ago
• tensorwavecloud / ScalarLM
  ScalarLM - a unified training and inference stack
  ☆93Updated last week
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆65Updated this week
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆299Updated this week
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆37Updated last month
• sunnweiwei / PPP-Agent
  ☆77Updated last week
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆259Updated this week
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆131Updated last month
• bentoml / llm-optimizer
  Benchmark and optimize LLM inference across frameworks with ease
  ☆131Updated 2 months ago
• run-ai / runai-model-streamer
  ☆267Updated this week
• perplexityai / pplx-garden
  Perplexity open source garden for inference technology
  ☆182Updated last week
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆266Updated last year
• aniketmaurya / fastserve-ai
  Machine Learning Serving focused on GenAI with simplicity as the top priority.
  ☆58Updated last month
• google-deepmind / asyncdiloco
  ☆47Updated last year
• InferenceMAX / InferenceMAX
  Open Source Continuous Inference Benchmarking - GB200 NVL72 vs MI355X vs B200 vs H200 vs MI325X & soon™ TPUv6e/v7/Trainium2/3/GB300 NVL72…
  ☆343Updated this week
• apple / ml-recurrent-drafter
  ☆218Updated 9 months ago
• opea-project / GenAIStudio
  GenAI Studio is a low code platform to enable users to construct, evaluate, and benchmark GenAI applications. The platform also provide c…
  ☆53Updated 2 months ago
• brendanhogan / picoDeepResearch
  ☆68Updated 5 months ago
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆58Updated 3 weeks ago
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆62Updated last month
• substratusai / vllm-docker
  ☆64Updated 7 months ago
• open-lm-engine / lm-engine
  LM engine is a library for pretraining/finetuning LLMs
  ☆74Updated last week
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆93Updated this week
• axolotl-ai-cloud / axolotl-cookbook
  ☆36Updated 3 months ago
• cgftinc / benchmax
  Framework-Agnostic RL Environments for LLM Fine-Tuning
  ☆38Updated last week
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆122Updated 9 months ago
• IST-DASLab / Quartet
  ☆106Updated 2 weeks ago
• argilla-io / argilla-cookbook
  Simple examples using Argilla tools to build AI
  ☆56Updated 11 months ago
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆44Updated this week