Auto-tuning for vllm. Getting the best performance out of your LLM deployment (vllm+guidellm+optuna)
☆65Sep 25, 2026Updated last month
Alternatives and similar repositories for auto-tuning-vllm
Users that are interested in auto-tuning-vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- helm charts for deploying models with llm-d☆32Updated this week
- Distributed inference configuration tools☆23Updated this week
- llm-d benchmark scripts and tooling☆78Updated this week
- Models as a Service☆75Oct 21, 2025Updated last year
- Starter kits for building and deploying AI agents. Run interactively locally or deploy to Red Hat OpenShift (including RHOAI) via OGX.☆40Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆1,757Updated this week
- Guide for installing and testing RHOAI Models as a Service in Red Hat AI☆21Updated this week
- AI quickstart that provides interactive dashboard to analyze AI Model Performance as well as Openshift metrics collected from Prometheus☆27Oct 28, 2026Updated last week
- Synthetic Data Generation Toolkit for LLMs☆170Oct 29, 2026Updated last week
- The AI Accelerator is a template project for setting up Red Hat OpenShift AI using GitOps☆70Oct 28, 2026Updated last week
- Sardeenz is a proof-of-concept application that allows you to load more than one model on a given GPU. It allows you to add more and more…☆65Aug 14, 2026Updated 2 months ago
- AI-on-OpenShift website source code☆109Oct 30, 2026Updated last week
- Repository to deploy LLMs with Multi-GPUs in distributed Kubernetes nodes☆30Dec 10, 2024Updated last year
- AI Gateway☆42Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Simplified model deployment on llm-d☆28Jul 2, 2025Updated last year
- An algorithm-focused interface for common llm training, continual learning, and reinforcement learning techniques☆104Oct 28, 2026Updated last week
- Artifacts for the Distributed Workloads stack as part of ODH☆36Updated this week
- ☆33Oct 28, 2026Updated last week
- A Single Helm Template for Deploying Many Application to Kubernetes (Mono Chart)☆10May 30, 2026Updated 5 months ago
- State of the art inference performance with modern accelerators on Kubernetes☆5,158Updated this week
- ☆59Aug 1, 2025Updated last year
- A file search engine based on modern search engine algorithms and data structures☆10May 14, 2018Updated 8 years ago
- 🎓The path to LLM mastery is paved with broken embeddings and resurrected gradients. (CHINESE NOTES)☆13Jul 16, 2026Updated 3 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Benchmark and optimize LLM inference across frameworks with ease☆203Oct 20, 2026Updated 2 weeks ago
- AI Agents Workshop with Red Hat AI☆14Feb 26, 2025Updated last year
- Vanilla configurations for a RHOAI instance to deploy a GenAI POC. This will deploy a vector database (Milvus), a GenAI interface (Anythi…☆14Apr 18, 2025Updated last year
- ☆33Oct 8, 2026Updated 3 weeks ago
- Operator for the OpenShift Lightspeed Service☆19Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆17Updated this week
- Unified adversarial verifier hardening loop☆16Jul 3, 2026Updated 4 months ago
- ☆18Aug 23, 2025Updated last year
- llm-d helm charts and deployment examples☆59May 1, 2026Updated 6 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- 从 nano-vllm 出发，一个 PR 一个 PR 地搓出 DeepInfer：边学边做推理引擎的 why 和 how☆26Updated this week
- LLM 推理服务性能测试☆46Dec 17, 2023Updated 2 years ago
- Header-only C++ library for writing PCP PMDAs☆16Feb 5, 2019Updated 7 years ago
- ☆137Updated this week
- Hand translated version of Cleveland's Lowess.c☆10Jun 22, 2023Updated 3 years ago
- ☆17Updated this week
- ☆14Updated this week