Alternatives and similar repositories for vllm-benchmark:

Users that are interested in vllm-benchmark are comparing it to the libraries listed below

• nyunAI / PruneGPT
  ☆53Updated 10 months ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆65Updated last year
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆136Updated 9 months ago
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆88Updated 11 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆86Updated last month
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆70Updated 2 months ago
• GreenBitAI / green-bit-llm
  A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.
  ☆82Updated last month
• unslothai / unsloth-zoo
  Utils for Unsloth
  ☆73Updated last week
• argilla-io / argilla-cookbook
  Simple examples using Argilla tools to build AI
  ☆52Updated 5 months ago
• IlyasMoutawwakil / py-txi
  A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
  ☆34Updated 4 months ago
• aniketmaurya / fastserve-ai
  Machine Learning Serving focused on GenAI with simplicity as the top priority.
  ☆58Updated 2 weeks ago
• arcee-ai / EvolKit
  EvolKit is an innovative framework designed to automatically enhance the complexity of instructions used for fine-tuning Large Language M…
  ☆213Updated 5 months ago
• h2oai / enterprise-h2ogpte
  Client Code Examples, Use Cases and Benchmarks for Enterprise h2oGPTe RAG-Based GenAI Platform
  ☆87Updated this week
• lightblue-tech / lb-reranker
  ☆24Updated 2 months ago
• kevaldekivadiya2415 / textembed
  TextEmbed is a REST API crafted for high-throughput and low-latency embedding inference. It accommodates a wide variety of embedding mode…
  ☆23Updated 7 months ago
• daniel-furman / sft-demos
  Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.
  ☆76Updated 6 months ago
• LLM360 / crystalcoder-data-prep
  Data preparation code for CrystalCoder 7B LLM
  ☆44Updated 11 months ago
• cognitivecomputations / spectrum
  ☆113Updated 2 weeks ago
• triton-inference-server / vllm_backend
  ☆246Updated last week
• bentoml / llm-bench
  ☆50Updated 5 months ago
• ComposioHQ / Composio-Function-Calling-Benchmark
  Function Calling Benchmark & Testing
  ☆87Updated 9 months ago
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆60Updated 4 months ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆262Updated 6 months ago
• vllm-project / dashboard
  vLLM performance dashboard
  ☆27Updated last year
• interstellarninja / function-calling-eval
  A framework for evaluating function calls made by LLMs
  ☆37Updated 9 months ago
• LLM-inference-router / vllm-router
  vLLM Router
  ☆26Updated last year
• substratusai / stapi
  Sentence Transformers API: An OpenAI compatible embedding API server
  ☆54Updated 7 months ago
• amogkam / llama_index_ray
  Using LlamaIndex with Ray for productionizing LLM applications
  ☆71Updated last year
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆106Updated last week
• run-ai / llmperf
  ☆53Updated 7 months ago