Alternatives and similar repositories for auto-tuning-vllm

Users that are interested in auto-tuning-vllm are comparing it to the libraries listed below

• openshift-psap / llm-load-test
  ☆50Updated 3 months ago
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆62Updated 2 months ago
• vllm-project / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆700Updated last week
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆65Updated last week
• llm-d / llm-d-benchmark
  llm-d benchmark scripts and tooling
  ☆33Updated this week
• run-ai / runai-model-streamer
  ☆267Updated this week
• bentoml / llm-bench
  ☆56Updated last year
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆83Updated 2 weeks ago
• llm-d / llm-d
  Achieve state of the art inference performance with modern accelerators on Kubernetes
  ☆2,026Updated last week
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆44Updated this week
• sgl-project / ome
  OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)
  ☆312Updated this week
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆266Updated last year
• foundation-model-stack / fms-hf-tuning
  🚀 Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.
  ☆52Updated last week
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆131Updated last month
• triton-inference-server / vllm_backend
  ☆312Updated this week
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆217Updated last year
• huggingface / inference-benchmarker
  Inference server benchmarking tool
  ☆128Updated last month
• kserve / modelmesh
  Distributed Model Serving Framework
  ☆178Updated last month
• microsoft / batch-inference
  Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
  ☆102Updated last year
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆44Updated this week
• qoofyk / LLM_Sizing_Guide
  A calculator to estimate the memory footprint, capacity, and latency on VMware Private AI with NVIDIA.
  ☆37Updated 3 months ago
• IBM / autopilot
  A tool to detect infrastructure issues on cloud native AI systems
  ☆51Updated 2 months ago
• llm-d / llm-d-deployer
  Helm charts for llm-d
  ☆50Updated 3 months ago
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆300Updated this week
• fmperf-project / fmperf
  Cloud Native Benchmarking of Foundation Models
  ☆44Updated 3 months ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆78Updated last year
• zipnn / zipnn
  A Lossless Compression Library for AI pipelines
  ☆286Updated 4 months ago
• ai-dynamo / grove
  Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling
  ☆110Updated this week
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 8 months ago
• clearml / clearml-fractional-gpu
  ClearML Fractional GPU - Run multiple containers on the same GPU with driver level memory limitation ✨ and compute time-slicing
  ☆82Updated this week