Alternatives and similar repositories for vllm:

Users that are interested in vllm are comparing it to the libraries listed below

• snowflakedb / ArcticTraining
  ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)
  ☆51Updated last week
• kevinwu23 / StanfordFineTuneBench
  ☆27Updated 4 months ago
• cray-lm / cray-lm
  Cray-LM unified training and inference stack.
  ☆21Updated last month
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆86Updated 2 months ago
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆40Updated last year
• Alignment-Lab-AI / datagen
  a pipeline for using api calls to agnostically convert unstructured data into structured training data
  ☆30Updated 6 months ago
• UmerHA / triton_util
  Make triton easier
  ☆47Updated 9 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆70Updated last month
• IBM / dolomite-engine
  Dolomite Engine is a library for pretraining/finetuning LLMs
  ☆44Updated this week
• tensorchord / inference-benchmark
  Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
  ☆28Updated last year
• determined-ai / determined-examples
  Example ML projects that use the Determined library.
  ☆29Updated 6 months ago
• google-deepmind / xtr
  XTR: Rethinking the Role of Token Retrieval in Multi-Vector Retrieval
  ☆49Updated 9 months ago
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated last year
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆25Updated this week
• drisspg / transformer_nuggets
  A place to store reusable transformer components of my own creation or found on the interwebs
  ☆48Updated this week
• eth-easl / fmengine
  Utilities for Training Very Large Models
  ☆58Updated 6 months ago
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆47Updated 6 months ago
• chainyo / tensorshare
  🤝 Trade any tensors over the network
  ☆30Updated last year
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"
  ☆59Updated 5 months ago
• trapoom555 / Language-Model-STS-CFT
  Improving Text Embedding of Language Models Using Contrastive Fine-tuning
  ☆61Updated 7 months ago
• BlackHC / llmtracer
  Trace LLM calls (and others) and visualize them in WandB, as interactive SVG or using a streaming local webapp
  ☆14Updated last month
• lianakoleva / no-libtorch-compile
  ☆21Updated 3 weeks ago
• google-deepmind / asyncdiloco
  ☆43Updated last year
• zhisbug / ray-scalable-ml-design
  Some microbenchmarks and design docs before commencement
  ☆12Updated 4 years ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆53Updated last month
• cortexlabs / nucleus
  Cortex-compatible model server for Python and TensorFlow
  ☆17Updated 2 years ago
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆35Updated 10 months ago
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆60Updated 3 months ago
• v-prgmr / mergekit
  Tools for merging pretrained large language models.
  ☆19Updated 9 months ago
• HazyResearch / aioli
  Aioli: A unified optimization framework for language model data mixing
  ☆22Updated 2 months ago