Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 6 months ago
• tensorwavecloud / ScalarLM
  ScalarLM - a unified training and inference stack
  ☆79Updated 2 weeks ago
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆42Updated this week
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆244Updated this week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆61Updated last week
• substratusai / vllm-docker
  ☆64Updated 6 months ago
• nexusflowai / NexusBench
  Nexusflow function call, tool use, and agent benchmarks.
  ☆29Updated 9 months ago
• cgftinc / benchmax
  Framework-Agnostic RL Environments for LLM Fine-Tuning
  ☆36Updated last week
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆226Updated this week
• opea-project / GenAIStudio
  GenAI Studio is a low code platform to enable users to construct, evaluate, and benchmark GenAI applications. The platform also provide c…
  ☆50Updated last month
• facebookresearch / matrix
  Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…
  ☆94Updated this week
• cray-lm / cray-lm
  Cray-LM unified training and inference stack.
  ☆22Updated 7 months ago
• raphaelmansuy / iteration_of_tought
  Example implementation of Iteration of Tought - Gives a star if you like the project
  ☆43Updated 9 months ago
• argilla-io / argilla-cookbook
  Simple examples using Argilla tools to build AI
  ☆55Updated 10 months ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆266Updated 11 months ago
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆48Updated this week
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆89Updated last week
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆36Updated 2 months ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆66Updated last year
• Cerebras / DocChat
  GPT-4 Level Conversational QA Trained In a Few Hours
  ☆64Updated last year
• bipul1010 / agents_tutorial
  ☆19Updated last year
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆150Updated 2 weeks ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆132Updated 3 months ago
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆55Updated 7 months ago
• substratusai / helm
  ☆18Updated last year
• QuixiAI / kraken
  ☆67Updated last year
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆72Updated last year
• brendanhogan / picoDeepResearch
  ☆68Updated 4 months ago
• shivamsanju / ragswift
  🚀 Scale your RAG pipeline using Ragswift: A scalable centralized embeddings management platform
  ☆38Updated last year
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆132Updated this week