Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• guidance-ai / jsonschemabench
  ☆75Updated 7 months ago
• groq / groqflow
  GroqFlow provides an automated tool flow for compiling machine learning and linear algebra workloads into Groq programs and executing tho…
  ☆114Updated 5 months ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆67Updated last year
• sayakpaul / hf-codegen
  A repository of Python scripts to scrape code contents of the public repositories of `huggingface`.
  ☆53Updated last year
• ibm-granite / granite-guardian
  The Granite Guardian models are designed to detect risks in prompts and responses.
  ☆128Updated 3 months ago
• ajtejankar / mixtral-vis-moe
  Visualize expert firing frequencies across sentences in the Mixtral MoE model
  ☆18Updated 2 years ago
• dottxt-ai / benchmarks
  Benchmark structured generation libraries
  ☆30Updated last year
• zhudotexe / redel
  ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)
  ☆90Updated last month
• weaviate / structured-rag
  Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models
  ☆114Updated 9 months ago
• nexusflowai / NexusBench
  Nexusflow function call, tool use, and agent benchmarks.
  ☆30Updated last year
• ibm-granite / granite-3.0-language-models
  ☆269Updated 7 months ago
• anarchy-ai / llm-speed-benchmark
  Benchmarking tool for assessing LLM models' performance across different hardwares
  ☆17Updated 2 years ago
• mistralai / vllm-release
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆53Updated 2 years ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆94Updated 10 months ago
• Aider-AI / aider-swe-bench
  Harness used to benchmark aider against SWE Bench benchmarks
  ☆78Updated last year
• METR / eval-analysis-public
  Public repository containing METR's DVC pipeline for eval data analysis
  ☆186Updated last week
• i-am-bee / beeai-code-interpreter
  An HTTP service intended as a backend for an LLM that can run arbitrary pieces of Python code.
  ☆70Updated 4 months ago
• qnixsynapse / rich-chat
  Python console application designed to provide an engaging and visually appealing LLM chat experience on Unix-like consoles or Terminals.
  ☆23Updated 3 weeks ago
• yandex-research / swarm
  Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"
  ☆149Updated 2 years ago
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆282Updated this week
• axolotl-ai-cloud / axolotl-cookbook
  ☆38Updated 5 months ago
• HishamAlyahya / semantic_backprop
  Source code of "How to Correctly do Semantic Backpropagation on Language-based Agentic Systems" 🤖
  ☆76Updated last year
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆93Updated last week
• groq / mlagility
  Machine Learning Agility (MLAgility) benchmark and benchmarking tools
  ☆40Updated 5 months ago
• voyage-ai / voyageai-python
  Voyage AI Official Python Library
  ☆91Updated last month
• ibm-granite / granite-3.1-language-models
  Granite 3.1 Language Models
  ☆136Updated 7 months ago
• JoshuaPurtell / SmallBench
  Small, simple agent task environments for training and evaluation
  ☆19Updated last year
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆133Updated last week
• NVIDIA / workbench-example-mistral-finetune
  An NVIDIA AI Workbench example project for fine-tuning a Mistral 7B model
  ☆69Updated last year
• skypilot-org / skypilot-tutorial
  Tutorial to get started with SkyPilot!
  ☆58Updated last year