Alternatives and similar repositories for guidellm

Users that are interested in guidellm are comparing it to the libraries listed below

• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆264Updated 9 months ago
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆219Updated 6 months ago
• triton-inference-server / vllm_backend
  ☆271Updated last month
• huggingface / inference-benchmarker
  Inference server benchmarking tool
  ☆79Updated 2 months ago
• vllm-project / llm-compressor
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
  ☆1,616Updated this week
• snowflakedb / ArcticInference
  ☆173Updated this week
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆349Updated 10 months ago
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆209Updated 11 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 4 months ago
• run-ai / runai-model-streamer
  ☆228Updated last week
• huggingface / optimum-benchmark
  🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…
  ☆305Updated last month
• efeslab / Nanoflow
  A throughput-oriented high-performance serving framework for LLMs
  ☆835Updated last month
• ray-project / llmperf
  LLMPerf is a library for validating and benchmarking LLMs
  ☆953Updated 7 months ago
• intel / auto-round
  Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU. Seamlessly integrated with Torchao, Tra…
  ☆526Updated this week
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆209Updated this week
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆76Updated this week
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆137Updated 11 months ago
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆647Updated 2 months ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆133Updated last week
• arcee-ai / DistillKit
  An Open Source Toolkit For LLM Distillation
  ☆669Updated last month
• snowflakedb / ArcticTraining
  ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)
  ☆151Updated this week
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆129Updated this week
• facebookresearch / LayerSkip
  Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
  ☆314Updated 2 months ago
• anyscale / llm-router
  Tutorial for building LLM router
  ☆214Updated 11 months ago
• ray-project / llmperf-leaderboard
  ☆464Updated last year
• bentoml / llm-bench
  ☆54Updated 7 months ago
• NVIDIA / Star-Attention
  Efficient LLM Inference over Long Sequences
  ☆382Updated 2 weeks ago
• apoorvumang / prompt-lookup-decoding
  ☆544Updated 10 months ago
• ibm-granite / granite-3.0-language-models
  ☆259Updated 2 weeks ago
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆198Updated 2 weeks ago