LLM inference decode throughput benchmark with Rich TUI dashboard. Measures token generation speed across concurrency levels and context lengths. Supports SGLang and vLLM engines.
☆53Jul 11, 2026Updated this week
Alternatives and similar repositories for llm-inference-bench
Users that are interested in llm-inference-bench are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆128Updated this week
- Docker images for LLM inference (SGLang + vLLM) on NVIDIA Blackwell GPUs (SM120, CUDA 13.2)☆37Updated this week
- RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink☆633Updated this week
- ☆32Apr 26, 2026Updated 2 months ago
- An electron Wrapper for Open-Interpreter for the lablab.ai hackathon☆12Oct 14, 2023Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆22May 30, 2026Updated last month
- Live dashboard: fire N parallel streaming coding-agent runs at any OpenAI-compatible endpoint — per-run TTFT/tok-s/E2E, real kill switch,…☆17Jul 8, 2026Updated last week
- DeepSeek-V4-Flash-DSpark abliterated (uncensored) · ~100% refusal bypass · C1 ~57 tok/s · 1M ctx · 2× DGX Spark · HF weights☆21Updated this week
- Agentic BYOK Browser-Based Website Builder☆51Updated this week
- AEON vLLM Ultimate — vLLM 0.25.0 built from source for DGX Spark / Blackwell (sm_121a/GB10). One image serves the whole AEON fleet (Gemma…☆71Updated this week
- These are performance benchmarks we did to prepare for our own privacy-preserving and NDA-compliant in-house AI coding assistant. If by a…☆31Apr 2, 2025Updated last year
- A plugin architecture for React.☆12May 10, 2021Updated 5 years ago
- Solve algorithmic Python challenges to sharpen the tools.☆11Apr 13, 2026Updated 3 months ago
- A simple mouse reporter☆19Apr 18, 2022Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Interactive offline OpenStreetMap viewer in the terminal.☆14Apr 9, 2018Updated 8 years ago
- ☆11Jul 24, 2025Updated 11 months ago
- Build different debian suites for the radxa rock 4 se☆12Mar 11, 2024Updated 2 years ago
- Tool-calling quality benchmark for LLM serving stacks. 80+ deterministic scenarios testing multi-turn orchestration, safety boundaries, a…☆196Jul 6, 2026Updated last week
- Emulation Station for Windows 10☆11Apr 26, 2021Updated 5 years ago
- Generate cordova/splash files from a single svg, and update config.xml☆12Feb 25, 2022Updated 4 years ago
- 📡 GPS satellite tracker - which GPS satellites are currently visible above a given location on earth?☆17Nov 6, 2019Updated 6 years ago
- Using Bayesian inference to mine rule sets☆12Jan 9, 2020Updated 6 years ago
- Get your current location from the OS.☆20Feb 7, 2018Updated 8 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- punt makes adb logcat better☆15Apr 28, 2022Updated 4 years ago
- ARCHIVED: Replaces your broccoli tmp/ folder w ram disk for faster builds (?) / minimal SSD thrashing☆41Oct 8, 2015Updated 10 years ago
- ☆14May 28, 2023Updated 3 years ago
- A <HeatMap /> Native Module component for React Native.☆19Jan 4, 2023Updated 3 years ago
- TokenSpeed is a speed-of-light LLM inference engine.☆1,589Updated this week
- 🤯💬 A Turing-Complete Binary Brainfuck Interpreter built only using CSS & HTML☆17May 15, 2024Updated 2 years ago
- Educational reference: NVIDIA Blackwell SM100 vs SM120, NVFP4, tcgen05, MoE inference on consumer Blackwell☆19Apr 28, 2026Updated 2 months ago
- 🕊️ Is it a bird? ✈️ Is it a plane? No, it's a search-bar-button! ⚛️☆22Sep 27, 2020Updated 5 years ago
- Agent-friendly GPU profile-query CLI☆105Jun 22, 2026Updated 3 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- TOPLOC: is a novel method for verifiable inference that enables users to verify that LLM providers are using the correct model configurat…☆57Updated this week
- Small program to run requests against a web server and look for problems☆11Jan 20, 2016Updated 10 years ago
- ASAP smoothing☆13Sep 8, 2017Updated 8 years ago
- A Go library for working with rows, columns, or matrix (deprecated, see https://github.com/shuLhan/share/tree/master/lib/tabula).☆11Nov 28, 2018Updated 7 years ago
- A library of parallel sparse preconditionersfor PSBLAS☆15Jun 16, 2026Updated 3 weeks ago
- Yowsup NodeJS, Python Middleware☆16Mar 31, 2019Updated 7 years ago
- Linux & Powershell scripts to easily set up and run the Qwen 3.5 series locally on Windows and Linux with llama.cpp.☆90Apr 28, 2026Updated 2 months ago