Benchmarking the serving capabilities of vLLM
☆59Aug 20, 2024Updated last year
Alternatives and similar repositories for vllm-benchmark
Users that are interested in vllm-benchmark are comparing it to the libraries listed below
Sorting:
- Automatic Thief Detection via CCTV with Alarm System and Perpetrator Image Capture using YOLOv5 + ROI. This project utilizes computer vis…☆14Oct 21, 2024Updated last year
- Nsight Compute In Docker☆13Dec 21, 2023Updated 2 years ago
- ☆14Jan 7, 2025Updated last year
- The driver for LMCache core to run in vLLM☆61Feb 4, 2025Updated last year
- Evaluate gpt-4o on CLIcK (Korean NLP Dataset)☆20May 18, 2024Updated last year
- Rust crate for some audio utilities☆27Mar 8, 2025Updated 11 months ago
- ☆19Nov 5, 2024Updated last year
- Verifiers for LLM Reinforcement Learning☆80Apr 15, 2025Updated 10 months ago
- bilibili黑马程序员Java企业级实战开发《学成在线》微服务项目（版次：2023-01-13）☆10Apr 4, 2023Updated 2 years ago
- KoCommonGEN v2: A Benchmark for Navigating Korean Commonsense Reasoning Challenges in Large Language Models☆25Aug 24, 2024Updated last year
- IBM development fork of https://github.com/huggingface/text-generation-inference☆63Sep 18, 2025Updated 5 months ago
- Open-source examples and guides for building with the Qwen. Browse a collection of snippets, advanced techniques and walkthroughs.☆37Nov 20, 2024Updated last year
- A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.☆32Sep 19, 2025Updated 5 months ago
- A Practitioner's Guide to M(eow)ti Turn Agentic ReinfOrcement learning☆75Jan 16, 2026Updated last month
- A high-throughput and memory-efficient inference and serving engine for Whisper, https://mesolitica.com/blog/vllm-whisper☆33Jul 28, 2024Updated last year
- ☆151Feb 26, 2026Updated last week
- hwpxlib 패키지 python에서 쉽게 사용 할수 있게 만든 github repo 입니다.☆36Mar 29, 2025Updated 11 months ago
- vLLM performance dashboard☆42Apr 26, 2024Updated last year
- Installer script tailored for Debian/Ubuntu systems to installs necessary packages.☆39Mar 7, 2024Updated last year
- This project is based on the [LTX-Video](https://github.com/Lightricks/LTX-Video) algorithm of the diffusers and optimized and accelerate…☆13Dec 31, 2024Updated last year
- FlappyBird愤怒的小鸟 c++游戏实现 学习代码☆10Nov 16, 2018Updated 7 years ago
- The best library in the world to generate PDF from HTML☆13Feb 24, 2026Updated last week
- An Offline and Secure Retrieval-Augmented Generation (RAG) system designed for efficient processing of diverse content types with minimal…☆20Dec 29, 2024Updated last year
- Getting Started with Analytics Engineering☆14Jul 25, 2024Updated last year
- This repo contains documentation related to the operation of the OpenBytes project.☆13Oct 29, 2021Updated 4 years ago
- Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization☆12Dec 3, 2024Updated last year
- ☆11Nov 10, 2020Updated 5 years ago
- Evaluation of Oasis Platform - simple install, UI and API☆14Feb 9, 2026Updated 3 weeks ago
- A web interface for SleekDB written in PHP☆11Jan 22, 2022Updated 4 years ago
- LLM-DSE: Searching Accelerator Parameters with LLM Agents☆13May 22, 2025Updated 9 months ago
- ☆17Aug 5, 2025Updated 7 months ago
- Application for Agent re-engineering for better and reliable Gen AI workflows.☆10Jul 20, 2025Updated 7 months ago
- Indexing framework designed for the automated creation of structured knowledge bases in Azure AI Search☆14Jun 18, 2025Updated 8 months ago
- ☆12Aug 6, 2024Updated last year
- EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU☆50Oct 6, 2024Updated last year
- [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention☆53Aug 6, 2025Updated 7 months ago
- Efficient Long-context Language Model Training by Core Attention Disaggregation☆92Updated this week
- 一个为RAG系统设计的Markdown文档工具，提供标题结构自动抽取和文档分割两大功能。完整保留文档层级结构，解决传统切分器丢失标题层级与破坏表格完整性的问题。A hierarchy-preserving Markdown document splitter for RAG…☆12Jan 2, 2025Updated last year
- A LaTeX template for Bachelor or Master theses☆12Jun 10, 2022Updated 3 years ago