Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆93Updated 9 months ago
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆14Updated 5 months ago
• exo-explore / gym
  EXO Gym is an open-source Python toolkit that facilitates distributed AI research.
  ☆87Updated 2 weeks ago
• guidance-ai / jsonschemabench
  ☆68Updated 6 months ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆67Updated last year
• anarchy-ai / llm-speed-benchmark
  Benchmarking tool for assessing LLM models' performance across different hardwares
  ☆17Updated 2 years ago
• groq / groqflow
  GroqFlow provides an automated tool flow for compiling machine learning and linear algebra workloads into Groq programs and executing tho…
  ☆114Updated 4 months ago
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆125Updated 10 months ago
• skypilot-org / skypilot-tutorial
  Tutorial to get started with SkyPilot!
  ☆58Updated last year
• open-lm-engine / lm-engine
  LM engine is a library for pretraining/finetuning LLMs
  ☆77Updated this week
• JoshuaPurtell / SmallBench
  Small, simple agent task environments for training and evaluation
  ☆19Updated last year
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆62Updated 3 months ago
• zhudotexe / redel
  ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)
  ☆89Updated last week
• friendliai / friendli-client
  [⛔️ DEPRECATED] Friendli: the fastest serving engine for generative AI
  ☆49Updated 5 months ago
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆94Updated this week
• i-am-bee / beeai-code-interpreter
  An HTTP service intended as a backend for an LLM that can run arbitrary pieces of Python code.
  ☆69Updated 3 months ago
• yandex-research / swarm
  Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"
  ☆148Updated 2 years ago
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆272Updated this week
• weaviate / structured-rag
  Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models
  ☆115Updated 8 months ago
• zipnn / zipnn
  A Lossless Compression Library for AI pipelines
  ☆289Updated 5 months ago
• sgl-project / sgl-project.github.io
  This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.
  ☆92Updated last week
• facebookresearch / fastgen
  Simple high-throughput inference library
  ☆152Updated 7 months ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆161Updated 3 weeks ago
• dropbox / aana_sdk
  Aana SDK is a powerful framework for building AI enabled multimodal applications.
  ☆54Updated 3 months ago
• QuixiAI / kraken
  ☆68Updated last year
• bentoml / llm-optimizer
  Benchmark and optimize LLM inference across frameworks with ease
  ☆150Updated 3 months ago
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆144Updated this week
• raphaelmansuy / iteration_of_tought
  Example implementation of Iteration of Tought - Gives a star if you like the project
  ☆41Updated 11 months ago
• sayakpaul / hf-codegen
  A repository of Python scripts to scrape code contents of the public repositories of `huggingface`.
  ☆53Updated last year
• IBM / unitxt
  🦄 Unitxt is a Python library for enterprise-grade evaluation of AI performance, offering the world's largest catalog of tools and data …
  ☆212Updated this week