Alternatives and similar repositories for vllm

Users that are interested in vllm are comparing it to the libraries listed below

• anarchy-ai / llm-speed-benchmark
  Benchmarking tool for assessing LLM models' performance across different hardwares
  ☆17Updated 2 years ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆68Updated last year
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆63Updated 4 months ago
• dropbox / aana_sdk
  Aana SDK is a powerful framework for building AI enabled multimodal applications.
  ☆55Updated 5 months ago
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆17Updated 7 months ago
• friendliai / friendli-client
  [⛔️ DEPRECATED] Friendli: the fastest serving engine for generative AI
  ☆49Updated 7 months ago
• nexusflowai / NexusBench
  Nexusflow function call, tool use, and agent benchmarks.
  ☆30Updated last year
• guidance-ai / jsonschemabench
  ☆76Updated 7 months ago
• skypilot-org / skypilot-tutorial
  Tutorial to get started with SkyPilot!
  ☆58Updated last year
• dottxt-ai / benchmarks
  Benchmark structured generation libraries
  ☆30Updated last year
• ajtejankar / mixtral-vis-moe
  Visualize expert firing frequencies across sentences in the Mixtral MoE model
  ☆18Updated 2 years ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆89Updated last week
• ibm-granite / granite-3.0-language-models
  ☆270Updated 7 months ago
• aorwall / moatless-testbeds
  Moatless Testbeds allows you to create isolated testbed environments in a Kubernetes cluster where you can apply code changes through git…
  ☆14Updated 10 months ago
• ibm-granite / granite-guardian
  The Granite Guardian models are designed to detect risks in prompts and responses.
  ☆130Updated 4 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆94Updated 11 months ago
• QuixiAI / kraken
  ☆68Updated last year
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆93Updated this week
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆49Updated this week
• zhudotexe / redel
  ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)
  ☆90Updated last month
• Aider-AI / aider-swe-bench
  Harness used to benchmark aider against SWE Bench benchmarks
  ☆79Updated last year
• JoshuaPurtell / SmallBench
  Small, simple agent task environments for training and evaluation
  ☆19Updated last year
• sgl-project / sgl-project.github.io
  This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang
  ☆100Updated this week
• groq / groqflow
  GroqFlow provides an automated tool flow for compiling machine learning and linear algebra workloads into Groq programs and executing tho…
  ☆115Updated 6 months ago
• arcee-ai / DAM
  ☆56Updated last year
• aymeric-roucher / agent_reasoning_benchmark
  🔧 Compare how Agent systems perform on several benchmarks. 📊🚀
  ☆103Updated 6 months ago
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆287Updated this week
• ScalingIntelligence / codemonkeys
  ☆59Updated last year
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆50Updated this week
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆134Updated 2 weeks ago