Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• tensorwavecloud / ScalarLM
  ScalarLM - a unified training and inference stack
  ☆95Updated 2 months ago
• bentoml / llm-optimizer
  Benchmark and optimize LLM inference across frameworks with ease
  ☆155Updated 4 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆94Updated 10 months ago
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆277Updated this week
• cray-lm / cray-lm
  Cray-LM unified training and inference stack.
  ☆22Updated 11 months ago
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆375Updated this week
• tilde-research / MoMoE-impl
  Memory optimized Mixture of Experts
  ☆72Updated 5 months ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆163Updated this week
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆37Updated 3 months ago
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆63Updated 4 months ago
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆166Updated this week
• apple / ml-recurrent-drafter
  ☆218Updated 11 months ago
• UCB-ADRS / ADRS
  AI-Driven Research Systems (ADRS)
  ☆117Updated last month
• brendanhogan / picoDeepResearch
  ☆68Updated 7 months ago
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆190Updated last week
• aniketmaurya / fastserve-ai
  Machine Learning Serving focused on GenAI with simplicity as the top priority.
  ☆59Updated 2 weeks ago
• perplexityai / pplx-garden
  Perplexity open source garden for inference technology
  ☆332Updated 3 weeks ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆267Updated last month
• InferenceMAX / InferenceMAX
  Open Source Continuous Inference Benchmarking - GB200 NVL72 vs MI355X vs B200 vs H200 vs MI325X & soon™ TPUv6e/v7/Trainium2/3/GB300 NVL72…
  ☆419Updated this week
• dropbox / aana_sdk
  Aana SDK is a powerful framework for building AI enabled multimodal applications.
  ☆55Updated 4 months ago
• substratusai / vllm-docker
  ☆67Updated 9 months ago
• zenrran4nlp / Awesome-LLM-Inference-Serving
  ☆47Updated 8 months ago
• argilla-io / argilla-cookbook
  Simple examples using Argilla tools to build AI
  ☆57Updated last year
• facebookresearch / matrix
  Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…
  ☆256Updated last week
• open-lm-engine / lm-engine
  LM engine is a library for pretraining/finetuning LLMs
  ☆110Updated last week
• run-ai / runai-model-streamer
  ☆274Updated this week
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆82Updated 10 months ago
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆59Updated 3 months ago
• raphaelmansuy / iteration_of_tought
  Example implementation of Iteration of Tought - Gives a star if you like the project
  ☆41Updated last year
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆47Updated this week