Auto-tuning for vllm. Getting the best performance out of your LLM deployment (vllm+guidellm+optuna)
☆64Sep 25, 2026Updated 3 weeks ago
Alternatives and similar repositories for auto-tuning-vllm
Users that are interested in auto-tuning-vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- helm charts for deploying models with llm-d☆32Oct 10, 2026Updated last week
- A Python-based tool, trained on the state-of-the-art Google Pegasus model, specializing in generating abstracts from given YouTube video …☆10Aug 6, 2023Updated 3 years ago
- llm-d benchmark scripts and tooling☆73Updated this week
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆1,699Updated this week
- Starter kits for building and deploying AI agents. Run interactively locally or deploy to Red Hat OpenShift (including RHOAI) via OGX.☆36Oct 10, 2026Updated last week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A performance testing and analysis automation framework☆16Updated this week
- ✨ GenAIOps Enablement Lab Instructions - https://rhoai-genaiops.github.io/lab-instructions☆17Oct 1, 2026Updated 2 weeks ago
- AI quickstart that provides interactive dashboard to analyze AI Model Performance as well as Openshift metrics collected from Prometheus☆27Jun 9, 2026Updated 4 months ago
- Synthetic Data Generation Toolkit for LLMs☆167Updated this week
- Models as a Service☆75Oct 21, 2025Updated 11 months ago
- The AI Accelerator is a template project for setting up Red Hat OpenShift AI using GitOps☆71Updated this week
- This is a git focused workshop!☆10Feb 15, 2023Updated 3 years ago
- Sardeenz is a proof-of-concept application that allows you to load more than one model on a given GPU. It allows you to add more and more…☆64Aug 14, 2026Updated 2 months ago
- ☆31Oct 7, 2026Updated last week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Simplified model deployment on llm-d☆28Jul 2, 2025Updated last year
- Vanilla configurations for a RHOAI instance to deploy a GenAI POC. This will deploy a vector database (Milvus), a GenAI interface (Anythi…☆14Apr 18, 2025Updated last year
- Benchmark and optimize LLM inference across frameworks with ease☆202Jul 14, 2026Updated 3 months ago
- ☆59Aug 1, 2025Updated last year
- Inference Platform Simulation☆42Updated this week
- AI-on-OpenShift website source code☆109Oct 8, 2026Updated last week
- Achieve state of the art inference performance with modern accelerators on Kubernetes☆4,854Updated this week
- A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual h…☆215Updated this week
- GenAI inference performance benchmarking tool☆265Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆17Mar 27, 2026Updated 6 months ago
- An algorithm-focused interface for common llm training, continual learning, and reinforcement learning techniques☆100Oct 7, 2026Updated last week
- Red Hat Ecosystem Engineering - Agentic Collections☆53Oct 9, 2026Updated last week
- Workshop prático sobre a plataforma de aplicações OpenShift (CaaS/PaaS)☆18Mar 27, 2020Updated 6 years ago
- Benchmark Suite Invocation Scripting☆11Mar 16, 2022Updated 4 years ago
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆888Updated this week
- This project makes running the InstructLab large language model (LLM) fine-tuning process easy and flexible on OpenShift☆27Aug 27, 2025Updated last year
- SPDK fork of nvme-cli. No longer supported - use standard nvme-cli with SPDK nvme CUSE instead. See https://spdk.io/doc/nvme.html#nvme_…☆15Apr 10, 2024Updated 2 years ago
- AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solu…☆792Updated this week
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Lola is able to package AI Context Modules or skills into a distributed package to be supported across multiple AI assistants. Think of y…☆132Updated this week
- ☆17Oct 25, 2023Updated 2 years ago
- ☆22Mar 11, 2026Updated 7 months ago
- htop.dev website☆13Nov 5, 2024Updated last year
- ☆24Nov 18, 2025Updated 11 months ago
- This is a clone of an SVN repository at http://pagecache-mangagement.googlecode.com/svn/trunk. It had been cloned by http://svn2github.co…☆10May 23, 2013Updated 13 years ago
- A high performance batching router optimises max throughput for text inference workload☆16Sep 6, 2023Updated 3 years ago