Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• facebookresearch / matrix
  Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…
  ☆60Updated last week
• snowflakedb / ArcticInference
  ☆99Updated last week
• guidance-ai / jsonschemabench
  ☆39Updated last month
• JoshuaPurtell / SmallBench
  Small, simple agent task environments for training and evaluation
  ☆18Updated 7 months ago
• i-am-bee / beeai-code-interpreter
  An HTTP service intended as a backend for an LLM that can run arbitrary pieces of Python code.
  ☆60Updated last month
• snowflakedb / ArcticTraining
  ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)
  ☆105Updated this week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆60Updated 3 weeks ago
• foundation-model-stack / fms-hf-tuning
  🚀 Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.
  ☆44Updated this week
• SohamGovande / podplex
  🦾💻🌐 distributed training & serverless inference at scale on RunPod
  ☆17Updated last year
• IBM / unitxt
  🦄 Unitxt is a Python library for enterprise-grade evaluation of AI performance, offering the world's largest catalog of tools and data …
  ☆196Updated this week
• cray-lm / cray-lm
  Cray-LM unified training and inference stack.
  ☆22Updated 4 months ago
• open-lm-engine / lm-engine
  LM engine is a library for pretraining/finetuning LLMs
  ☆56Updated last week
• dottxt-ai / benchmarks
  Benchmark structured generation libraries
  ☆27Updated 7 months ago
• zhudotexe / redel
  ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)
  ☆78Updated 2 months ago
• bentoml / BentoLMDeploy
  Self-host LLMs with LMDeploy and BentoML
  ☆19Updated 2 months ago
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆42Updated this week
• AI-Hypercomputer / jetstream-pytorch
  PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"
  ☆60Updated 2 months ago
• LLMSELECTOR / LLMSELECTOR
  ☆68Updated 3 months ago
• l4b4r4b4b4 / AIDocks
  LLM-Training-API: Including Embeddings & ReRankers, mergekit, LaserRMT
  ☆27Updated last year
• kevinwu23 / StanfordFineTuneBench
  ☆29Updated 6 months ago
• JoshuaPurtell / LRCBench
  Evals meant to evaluate language models' ability to reason over long contexts.
  ☆9Updated 8 months ago
• nicknochnack / beeagent
  ☆12Updated 8 months ago
• brendanhogan / picoDeepResearch
  ☆59Updated 2 weeks ago
• run-ai / runai-model-streamer
  ☆215Updated this week
• flowaicom / flow-judge
  Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…
  ☆70Updated 7 months ago
• i-am-bee / bee-observe
  Observability API server for bee-agent-framework
  ☆13Updated 2 months ago
• huggingface / inference-benchmarker
  Inference server benchmarking tool
  ☆68Updated last month
• Nero10578 / LLM-Inference-Benchmark
  ☆14Updated 9 months ago
• raphaelmansuy / iteration_of_tought
  Example implementation of Iteration of Tought - Gives a star if you like the project
  ☆41Updated 5 months ago
• eqimp / hogwild_llm
  Official PyTorch implementation for Hogwild! Inference: Parallel LLM Generation with a Concurrent Attention Cache
  ☆105Updated last month