Interactive launcher and benchmarking harness for llama.cpp server throughput, with tests, sweeps, and round‑robin load tools.
☆482Feb 8, 2026Updated 9 months ago
Alternatives and similar repositories for llama-throughput-lab
Users that are interested in llama-throughput-lab are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆332Apr 13, 2026Updated 7 months ago
- ☆242Feb 1, 2026Updated 9 months ago
- A long-context code retrieval and reproduction benchmark.☆311Sep 4, 2026Updated 2 months ago
- llama-benchy - llama-bench style benchmarking tool for all backends☆777Jul 10, 2026Updated 4 months ago
- ☆2,027Nov 7, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Model Shelf is a local-first model resolver that helps AI agents and scripts find model weights on your own storage before downloading fr…☆128Sep 1, 2026Updated 2 months ago
- Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don…☆6,258Updated this week
- Use key or voice commands to direct your voice to any app or service you require, local or remote.☆31Updated this week
- ☆551Nov 2, 2026Updated last week
- MCP server that analyzes codebases and generates documentation for you☆16Oct 4, 2025Updated last year
- Fork of llama.cpp Nvida Volta V100 (tensor parallelism 4 x v100 GPU)☆12Updated this week
- A simple python-based terminal tool to manage llama.cpp☆13Sep 30, 2025Updated last year
- I Ran Qwen 3.8 27B on Just 4GB VRAM! (FreeToken Tutorial) - Edge-native MoE serving engine for local LLMs on low VRAM GPUs☆16Aug 23, 2026Updated 2 months ago
- ☆43Sep 22, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- This project demonstrates the application of Hybrid Markov Logic Networks (MLN) to verify and refine embeddings from the Cora citation ne…☆16Apr 18, 2025Updated last year
- Tile primitives for speedy kernels☆21Sep 13, 2026Updated 2 months ago
- simple script to read a corrected battery percentage on the beepy / beepberry☆14Aug 3, 2023Updated 3 years ago
- ESP32-C3/S3 firmware that polls hl-hostmon over USB CDC, renders host stats on a 1.54″ e-ink (GxEPD2), supports touch/auto modes, OTA (Ar…☆15Sep 5, 2025Updated last year
- Docker configuration for running VLLM on dual DGX Sparks☆2,499Nov 7, 2026Updated last week
- Hands-On Artificial Intelligence for Search, published by Packt☆13Jan 30, 2023Updated 3 years ago
- A local-first studio for generating and comparing AI videos through OpenRouter video models.☆27Apr 29, 2026Updated 6 months ago
- Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https…☆6,218Updated this week
- LLM inference in C/C++☆135,164Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A user-friendly GUI for llama.cpp — convert, quantize, and run GGUF models without touching the terminal.☆25Updated this week
- A Hebrew Analytical Lexicon based on ETCBC (4c) data☆12Oct 1, 2019Updated 7 years ago
- ☆1,707Updated this week
- MOTU midi express 128 linux driver☆23Jan 13, 2026Updated 10 months ago
- ☆223Updated this week
- A dynamic multi-expert AI architecture running on a single consumer GPU (RTX 3060).☆37Dec 2, 2025Updated 11 months ago
- 🚀 FlexLLama - Lightweight self-hosted tool for running multiple llama.cpp server instances with OpenAI v1 API compatibility and multi-GP…☆63Sep 26, 2026Updated last month
- ☆177Apr 7, 2026Updated 7 months ago
- Tiny, Apache-2.0 wake-word detection you train yourself and run on-device in Python, the browser, and on mobile.☆16Sep 30, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- RabbitMQ cluster formation using single Docker Swarm service☆11Feb 8, 2022Updated 4 years ago
- A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from…☆51,273Updated this week
- KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44…☆1,059Apr 23, 2026Updated 6 months ago
- Run frontier AI locally.☆48,434Nov 6, 2026Updated last week
- Enklayve is a secure, privacy-first desktop app that lets you ask your own documents anything, answered by AI models running entirely on …☆12Oct 26, 2026Updated 3 weeks ago
- ☆182Sep 2, 2026Updated 2 months ago
- Generate a conversation between two models using OpenAI API and Kokoro TTS.☆12May 12, 2025Updated last year
Are these results useful?