Interactive launcher and benchmarking harness for llama.cpp server throughput, with tests, sweeps, and round‑robin load tools.
☆474Feb 8, 2026Updated 8 months ago
Alternatives and similar repositories for llama-throughput-lab
Users that are interested in llama-throughput-lab are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A long-context code retrieval and reproduction benchmark.☆310Sep 4, 2026Updated last month
- llama-benchy - llama-bench style benchmarking tool for all backends☆762Jul 10, 2026Updated 3 months ago
- Use key or voice commands to direct your voice to any app or service you require, local or remote.☆29Oct 17, 2026Updated last week
- Bash script that keeps llama.cpp up to date: downloads the latest nightly b[NUM] binaries for Linux and macOS (x64 and ARM64, Vulkan on U…☆18Updated this week
- ☆2,003Oct 17, 2026Updated last week
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Model Shelf is a local-first model resolver that helps AI agents and scripts find model weights on your own storage before downloading fr…☆129Sep 1, 2026Updated last month
- VENDORIZED in lucebox-hub. Fork of llama.cpp, ggml graph for lucebox inference engine☆30Jul 8, 2026Updated 3 months ago
- ☆539Updated this week
- Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc☆6,076Updated this week
- ☆14Oct 17, 2026Updated last week
- Tile primitives for speedy kernels☆21Sep 13, 2026Updated last month
- SGLang for cool dudes using V100☆80Oct 5, 2026Updated 3 weeks ago
- A simple python-based terminal tool to manage llama.cpp☆13Sep 30, 2025Updated last year
- ☆43Sep 22, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Java library for joining objects into a graph, JOin gRAPH☆11Dec 12, 2020Updated 5 years ago
- Interactive AI storytelling app for Windows & Android with browser multiplayer, and local or API models.☆16Oct 15, 2026Updated last week
- A local-first studio for generating and comparing AI videos through OpenRouter video models.☆28Apr 29, 2026Updated 5 months ago
- Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https…☆5,993Updated this week
- MCP server to manage your GitHub starred repositories and lists☆16Jul 31, 2026Updated 2 months ago
- Highly specialized crate to parse and use `google/sentencepiece` 's precompiled_charsmap in `tokenizers`☆23Sep 23, 2026Updated last month
- LLM inference in C/C++☆132,320Updated this week
- Container toolbox that runs llama.cpp with AMD ROCm and NVIDIA CUDA in one build, for inference on two GPUs☆20Oct 16, 2026Updated last week
- Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦☆43,819Updated this week
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- A user-friendly GUI for llama.cpp — convert, quantize, and run GGUF models without touching the terminal.☆25Jun 9, 2026Updated 4 months ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆251Sep 14, 2026Updated last month
- Run your own AI cluster at home with everyday devices 📱💻 🖥️⌚☆22Jul 14, 2025Updated last year
- OpenClaw / Home Assistant voice assistant for the Voice PE — realtime conversation, custom wake words, speaker recognition, voice memory,…☆93Updated this week
- 🚀 FlexLLama - Lightweight self-hosted tool for running multiple llama.cpp server instances with OpenAI v1 API compatibility and multi-GP…☆62Sep 26, 2026Updated 3 weeks ago
- A dynamic multi-expert AI architecture running on a single consumer GPU (RTX 3060).☆37Dec 2, 2025Updated 10 months ago
- KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44…☆1,053Apr 23, 2026Updated 6 months ago
- ☆177Apr 7, 2026Updated 6 months ago
- One-click LLM server with TurboQuant Llama CPP engine☆17Apr 16, 2026Updated 6 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆171Sep 2, 2026Updated last month
- llama.cpp fork with additional SOTA quants and improved performance☆3,337Updated this week
- A comprehensive WebUI Toolkit for Resemble-AI's Chatterbox☆29Jun 7, 2025Updated last year
- Generate a conversation between two models using OpenAI API and Kokoro TTS.☆12May 12, 2025Updated last year
- DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm☆24,071Updated this week
- Rust terminal monitor for NVIDIA GPUs, Linux systems, and DGX Spark, with per-process memory views and persistent GPU job history.☆98Updated this week
- ☆103Mar 8, 2026Updated 7 months ago