Alternatives and similar repositories for llmperf-leaderboard:

Users that are interested in llmperf-leaderboard are comparing it to the libraries listed below

• ray-project / llmperf
  LLMPerf is a library for validating and benchmarking LLMs
  ☆710Updated last month
• apoorvumang / prompt-lookup-decoding
  ☆497Updated 5 months ago
• nexusflowai / NexusRaven
  NexusRaven-13B, a new SOTA Open-Source LLM for function calling. This repo contains everything for reproducing our evaluation on NexusRav…
  ☆311Updated last year
• punica-ai / punica
  Serving multiple LoRA finetuned LLM as one
  ☆1,018Updated 8 months ago
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆203Updated last month
• vllm-project / llm-compressor
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
  ☆894Updated this week
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆185Updated last month
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆352Updated 5 months ago
• mlabonne / llm-autoeval
  Automatically evaluate your LLMs in Google Colab
  ☆584Updated 8 months ago
• triton-inference-server / vllm_backend
  ☆218Updated this week
• sabetAI / BLoRA
  batched loras
  ☆338Updated last year
• ray-project / ray-llm
  RayLLM - LLMs on Ray
  ☆1,250Updated 8 months ago
• efeslab / Nanoflow
  A throughput-oriented high-performance serving framework for LLMs
  ☆714Updated 4 months ago
• mlc-ai / xgrammar
  Efficient, Flexible and Portable Structured Generation
  ☆619Updated this week
• mobiusml / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆737Updated 2 weeks ago
• jondurbin / bagel
  A bagel, with everything.
  ☆315Updated 9 months ago
• anyscale / llm-router
  Tutorial for building LLM router
  ☆173Updated 6 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆64Updated last month
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆76Updated 4 months ago
• huggingface / optimum-benchmark
  🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…
  ☆284Updated this week
• run-ai / llmperf
  ☆52Updated 4 months ago
• Preemo-Inc / text-generation-inference
  ☆199Updated 11 months ago
• willccbb / mlx_parallm
  Fast parallel LLM inference for MLX
  ☆153Updated 6 months ago
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆104Updated 7 months ago
• scaleapi / llm-engine
  Scale LLM Engine public repository
  ☆789Updated this week
• SqueezeAILab / LLMCompiler
  [ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling
  ☆1,590Updated 6 months ago
• tomaarsen / attention_sinks
  Extend existing LLMs way beyond the original training length with constant memory usage, without retraining
  ☆687Updated 9 months ago
• SkunkworksAI / hydra-moe
  ☆412Updated last year
• arthur-ai / bench
  A tool for evaluating LLMs
  ☆400Updated 8 months ago