Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• anarchy-ai / llm-speed-benchmark
  Benchmarking tool for assessing LLM models' performance across different hardwares
  ☆17Updated last year
• groq / mlagility
  Machine Learning Agility (MLAgility) benchmark and benchmarking tools
  ☆40Updated 2 months ago
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆90Updated last week
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 7 months ago
• guidance-ai / jsonschemabench
  ☆62Updated 4 months ago
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆61Updated last month
• nexusflowai / NexusBench
  Nexusflow function call, tool use, and agent benchmarks.
  ☆29Updated 10 months ago
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆120Updated 8 months ago
• groq / groqflow
  GroqFlow provides an automated tool flow for compiling machine learning and linear algebra workloads into Groq programs and executing tho…
  ☆112Updated 2 months ago
• facebookresearch / matrix
  Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…
  ☆96Updated last week
• WhitzardIndex / self-replication-research
  A preprint version of our recent research on the capability of frontier AI systems to do self-replication
  ☆57Updated 10 months ago
• mistralai / vllm-release
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆52Updated last year
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆41Updated this week
• isEmmanuelOlowe / llm-cost-estimator
  Estimating hardware and cloud costs of LLMs and transformer projects
  ☆18Updated 3 months ago
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆252Updated this week
• tensorwavecloud / ScalarLM
  ScalarLM - a unified training and inference stack
  ☆85Updated 2 weeks ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆134Updated 4 months ago
• ibm-granite / granite-3.1-language-models
  Granite 3.1 Language Models
  ☆128Updated 3 months ago
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆60Updated this week
• EQ-bench / eqbench3
  ☆26Updated 2 months ago
• JoshuaPurtell / SmallBench
  Small, simple agent task environments for training and evaluation
  ☆18Updated 11 months ago
• weaviate / structured-rag
  Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models
  ☆111Updated 6 months ago
• exo-explore / gym
  EXO Gym is an open-source Python toolkit that facilitates distributed AI research.
  ☆78Updated last month
• abacusai / gh200-llm
  Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning
  ☆50Updated 7 months ago
• mobiusml / aana_sdk
  Aana SDK is a powerful framework for building AI enabled multimodal applications.
  ☆52Updated last month
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆83Updated last week
• huseinzol05 / transformers-continuous-batching
  Lightweight continuous batching OpenAI compatibility using HuggingFace Transformers include T5 and Whisper.
  ☆29Updated 7 months ago
• ibm-granite / granite-guardian
  The Granite Guardian models are designed to detect risks in prompts and responses.
  ☆119Updated last week
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆14Updated 3 months ago
• arcee-ai / DAM
  ☆55Updated 11 months ago