Interactive launcher and benchmarking harness for llama.cpp server throughput, with tests, sweeps, and round‑robin load tools.
☆465Feb 8, 2026Updated 7 months ago
Alternatives and similar repositories for llama-throughput-lab
Users that are interested in llama-throughput-lab are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆326Apr 13, 2026Updated 5 months ago
- ☆232Feb 1, 2026Updated 7 months ago
- A long-context code retrieval and reproduction benchmark.☆304Sep 4, 2026Updated last week
- llama-benchy - llama-bench style benchmarking tool for all backends☆710Jul 10, 2026Updated 2 months ago
- ☆1,936Sep 5, 2026Updated last week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- VENDORIZED in lucebox-hub. Fork of llama.cpp, ggml graph for lucebox inference engine☆31Jul 8, 2026Updated 2 months ago
- Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc☆5,665Updated this week
- Model Shelf is a local-first model resolver that helps AI agents and scripts find model weights on your own storage before downloading fr…☆130Sep 1, 2026Updated last week
- ☆521Sep 2, 2026Updated last week
- ☆42Sep 22, 2025Updated 11 months ago
- This is a FastAPI based LLM server. Load multiple LLM models (MLX or llama.cpp) simultaneously using multiprocessing.☆19Apr 8, 2026Updated 5 months ago
- A Hebrew Analytical Lexicon based on ETCBC (4c) data☆12Oct 1, 2019Updated 6 years ago
- ☆20Jul 21, 2026Updated last month
- Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https…☆5,707Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Docker configuration for running VLLM on dual DGX Sparks☆2,271Updated this week
- LLM inference in C/C++☆128,196Updated this week
- 🚀 FlexLLama - Lightweight self-hosted tool for running multiple llama.cpp server instances with OpenAI v1 API compatibility and multi-GP…☆60Jun 10, 2026Updated 3 months ago
- Don't bug your friends with articles they'll never read. AI's have infinite attention, leverage them instead! Use the curation buddy to e…☆21May 2, 2024Updated 2 years ago
- KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44…☆1,047Apr 23, 2026Updated 4 months ago
- A comprehensive WebUI Toolkit for Resemble-AI's Chatterbox☆29Jun 7, 2025Updated last year
- Scripts and tools for optimizing quantizations in llama.cpp with GGUF imatrices.☆19Jan 10, 2025Updated last year
- Suckless no dependencies CUDA/C++ port of NVIDIA's DVLT, feed it images, get a 3D point cloud + camera poses. NO python. One fast 5MB bin…☆65Jun 4, 2026Updated 3 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆3,221Updated this week
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- ☆102Mar 8, 2026Updated 6 months ago
- ☆15Feb 4, 2021Updated 5 years ago
- Sonic - an OpenAI WebSocket-Mode Compatible Gateway for vLLM☆35Feb 24, 2026Updated 6 months ago
- Run frontier AI locally.☆47,424Aug 25, 2026Updated 2 weeks ago
- LLM inference in C/C++☆64May 7, 2026Updated 4 months ago
- Septuagint database based on Swete's text (1909-1930)☆21Jun 17, 2017Updated 9 years ago
- `Paper repo for “Coherence-Guided Dead-Head Identification in Frozen Transformers,” including manuscript sources, figures, frozen result …☆51Apr 8, 2026Updated 5 months ago
- Spreadsheet-like programming on all your devices. http://object.network/onex-app.html☆16Jun 17, 2025Updated last year
- ☆254Oct 30, 2025Updated 10 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Diffusion_TTS extension for booga☆71Sep 6, 2025Updated last year
- Bitcoin utilities and protocol library for interacting with the network☆16Oct 27, 2025Updated 10 months ago
- High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal mode…☆1,570Sep 6, 2026Updated last week
- A Python-powered RPG Engine☆12May 28, 2015Updated 11 years ago
- Minimal code to train ELMo models in recent versions of TensorFlow☆14Jun 16, 2026Updated 2 months ago
- High-performance GPU kernels for LLM inference in OpenAI Triton. Fused RMSNorm, SwiGLU, INT8 GEMM with benchmarks and roofline analysis.☆43Jul 22, 2026Updated last month
- Resources related to EMNLP 2021 paper "FAME: Feature-Based Adversarial Meta-Embeddings for Robust Input Representations"☆13Dec 14, 2021Updated 4 years ago