LLM inference in C/C++
☆473Nov 7, 2026Updated this week
Alternatives and similar repositories for llama.cpp-adaptive-kv-streaming
Users that are interested in llama.cpp-adaptive-kv-streaming are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.☆1,278Updated this week
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,187Updated this week
- ☆24Sep 20, 2026Updated last month
- Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell☆744Updated this week
- LLM inference in C/C++☆148Oct 16, 2026Updated 3 weeks ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith …☆906Updated this week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆41,381Updated this week
- A set of performant RDNA/HIP patches against llama.cpp focusing on high speed inferencing with numerics assurance☆236Updated this week
- ☆103Updated this week
- NInfer for Windows and around 16GB VRAM: RTX 5070 Ti / 5080 / 5090, Qwen3.8-27B GSQ-RCO Q3, CUDA 13 Native engine, tray manager, model co…☆64Oct 1, 2026Updated last month
- Qwen3.8 Local Coding on a 16GB GPU☆27Sep 18, 2026Updated last month
- RouteWeaver: experimental cache-safe speculative inference and CPU/GPU transfer optimization for Qwen 27B on low-VRAM NVIDIA GPUs.☆34Oct 10, 2026Updated 3 weeks ago
- llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ to…☆267Updated this week
- Experimental llama.cpp fork for inference research and development☆904Updated this week
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~4…☆479Updated this week
- Custom llama.cpp fork for running EschaLabs Escha models.☆13Sep 11, 2026Updated last month
- Pure-Rust MoE inference engine for Qwen3.5/3.6/3.8 GGUF. CPU+GPU hybrid experts, NVRTC CUDA kernels, OpenAI-compatible server. 2x llama.c…☆131Updated this week
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,283Updated this week
- Optimized NINFER runtime and reproducible Qwen3.8-27B true-128K inference for NVIDIA RTX 5080 16GB☆49Oct 23, 2026Updated 2 weeks ago
- Run MoE models beyond VRAM on AMD, NVIDIA & Intel GPUs. Vulkan-powered, portable Windows app with automatic setup. | 让游戏显卡跑起超显存大模型，支持 AMD…☆353Updated this week
- An agent / harness multiplexer that allows you to access your agents from any device and machine☆23Updated this week
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,800Updated this week
- One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF…☆201Updated this week
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- More MoEs: a fork of Strata (Qwen3.8-Flash-Next, Qwen3.6/Ornith and more on one GPU + system RAM)☆25Updated this week
- Run GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- …☆2,632Updated this week
- Source-available C++/CUDA inference engine for Qwen3.8-27B INT4 on 3x RTX 3090☆43Updated this week
- C++/CUDA single-GPU inference engine for Qwen3.8 (incl. Qwen3.8-Flash-Next with offloaded experts) on the RTX 5090; grew from NInfer☆280Updated this week
- ninfer-ext: Forked for speed.☆26Oct 22, 2026Updated 2 weeks ago
- Toolkit for optimizing or building inference engines tailored to your hardware and desired LLM.☆44Updated this week
- Local, gradient-free neuro-symbolic memory engine combining Hyperdimensional Computing (HDC/VSA), Hebbian plasticity, and graph triples f…☆314Updated this week
- LLM inference in C/C++☆2,445Oct 22, 2026Updated 2 weeks ago
- ☆11Jun 1, 2026Updated 5 months ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% t…☆432Oct 28, 2026Updated last week
- FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.☆16,798Updated this week
- Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when …☆2,601Updated this week
- Mirai S (Qwen3.8-27B-S trellis GGUF) on a 12 GB RTX 4070: 262k q8_0 tiered cache, MTP drafting at every depth, harness-proofing, the laye…☆109Updated this week
- ☆27Oct 13, 2026Updated 3 weeks ago
- Qwen3.8-Flash-Next at full 262k context on 8 GB RAM and 16 GB VRAM—20 tok/s generation and 1,600 tok/s prefill.☆65Oct 26, 2026Updated last week
- A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot int…☆665Updated this week