Alternatives and similar repositories for auto-tuning-vllm

Users that are interested in auto-tuning-vllm are comparing it to the libraries listed below

• openshift-psap / llm-load-test
  ☆51Updated 5 months ago
• foundation-model-stack / fms-hf-tuning
  🚀 Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.
  ☆56Updated last month
• vllm-project / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆819Updated this week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆63Updated 4 months ago
• instructlab / taxonomy
  Taxonomy tree that will allow you to create models tuned with your data
  ☆290Updated 4 months ago
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆47Updated this week
• instructlab / sdg
  Python library for Synthetic Data Generation
  ☆52Updated last month
• run-ai / runai-model-streamer
  ☆278Updated last week
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆50Updated this week
• instructlab / instructlab-bot
  GitHub bot to assist with the taxonomy contribution workflow
  ☆17Updated last year
• containers / ai-lab-recipes
  Examples for building and running LLM services and applications locally with Podman
  ☆190Updated 5 months ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆167Updated last week
• Red-Hat-AI-Innovation-Team / sdg_hub
  Synthetic Data Generation Toolkit for LLMs
  ☆88Updated last week
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆171Updated 2 weeks ago
• huggingface / inference-benchmarker
  Inference server benchmarking tool
  ☆141Updated 3 months ago
• instructlab / eval
  Python library for Evaluation
  ☆16Updated this week
• llm-d / llm-d-benchmark
  llm-d benchmark scripts and tooling
  ☆42Updated this week
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆94Updated 10 months ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆78Updated last year
• ai-dynamo / grove
  Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling
  ☆155Updated last week
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆205Updated last week
• kserve / modelmesh
  Distributed Model Serving Framework
  ☆182Updated 4 months ago
• clearml / clearml-fractional-gpu
  ClearML Fractional GPU - Run multiple containers on the same GPU with driver level memory limitation ✨ and compute time-slicing
  ☆88Updated 2 months ago
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆282Updated this week
• sgl-project / ome
  Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, T…
  ☆365Updated this week
• opea-project / GenAIStudio
  GenAI Studio is a low code platform to enable users to construct, evaluate, and benchmark GenAI applications. The platform also provide c…
  ☆58Updated 2 weeks ago
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆379Updated last week
• bentoml / llm-bench
  ☆56Updated last year
• opea-project / GenAIComps
  GenAI components at micro-service level; GenAI service composer to create mega-service
  ☆193Updated last week
• williamcaban / ollama-ubi
  ☆23Updated 10 months ago