Fused TBQ4 Flash Attention + MTP + Shared Tensors for llama.cpp — 82+ tok/s with lossless 4.25 bpv KV cache at 200K context on RTX 4090
☆89May 17, 2026Updated 2 months ago
Alternatives and similar repositories for llama.cpp-turboq-mtp
Users that are interested in llama.cpp-turboq-mtp are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆142Jun 13, 2026Updated last month
- LLM inference in C/C++☆56Updated this week
- llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% t…☆306Updated this week
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆789Updated this week
- LLM inference in C/C++☆2,152Updated this week
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- LLM inference in C/C++☆64May 7, 2026Updated 2 months ago
- LLAMA Turboquant implementation with CUDA support☆703Updated this week
- llama.cpp fork with additional SOTA quants and improved performance☆2,943Updated this week
- TurboQuant llama.cpp fork with optimized turbo4 kernels for Gemma 4 D=256/512 heads — lazy K/V, batch decode, warp-cooperative write. 120…☆35Apr 5, 2026Updated 3 months ago
- One-click Qwen3.6-27B inference on Windows. 158 tok/s on RTX 5090, 72 tok/s on RTX 3090. Native, no WSL, no Docker, no telemetry.☆222May 14, 2026Updated 2 months ago
- llama.cpp fork with TQ3_1S/4S CUDA kernels — 3.5-bit WHT quantization achieving Q4s quality at 10% smaller size. Based on RaBitQ-inspired…☆221Jul 6, 2026Updated 2 weeks ago
- LLM inference in C/C++☆36Apr 12, 2026Updated 3 months ago
- Easy install Autodesk products on Fedora☆12Aug 18, 2020Updated 5 years ago
- NEW ROCmfp4 format for llama.cpp☆135Jun 13, 2026Updated last month
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Patched native-Windows build of vLLM. Three Windows-specific fixes (CPU-relay for Gloo, Qwen3 reasoning parser, wildcard model name) on t…☆26May 8, 2026Updated 2 months ago
- ☆12Feb 19, 2025Updated last year
- Model-agnostic code memory MCP server. Budget-aware graph retrieval for AI agents. Sub-millisecond queries, token budgeting, deterministi…☆23May 18, 2026Updated 2 months ago
- Experimental LLM inference in C/C++☆39May 15, 2026Updated 2 months ago
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆386Updated this week
- Fast LLM speculative inference server for consumer hardware.☆2,668Updated this week
- AdaLLM is an NVFP4-first inference runtime for Ada Lovelace (RTX 4090) with FP8 KV cache and custom decode kernels. This repo targets NVF…☆135Feb 15, 2026Updated 5 months ago
- Usage indicator extension for pi with footer status bars and /usage command☆15Apr 17, 2026Updated 3 months ago
- The complete NUMA-optimized branch of the ktransformers project☆25Nov 3, 2025Updated 8 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- C# DDE Client for MetaTrader 4 (via Ndde)☆10Jan 1, 2018Updated 8 years ago
- A fast, lightweight HTML to Markdown converter optimized for LLM consumption. Uses proven parsing libraries to deliver clean, well-struct…☆77Updated this week
- Adaptive Precision for EXpert Models: MoE-aware mixed-precision quantization☆396May 29, 2026Updated last month
- 基于Ant Design Mobile 的Taro组件库 多端支持微信小程序、H5等☆11Sep 26, 2018Updated 7 years ago
- Multimodal AI studio powered by Qwen3.6-35B-A3B. End-to-end web app exposing visual reasoning, image captioning, and document understandi…☆27Apr 23, 2026Updated 2 months ago
- TouchDesigner > JSONLiveLink > Unreal Engine☆13Mar 31, 2022Updated 4 years ago
- Testbench for llama.cpp llama-server☆15Aug 20, 2025Updated 11 months ago
- A llamacpp wrapper to manage and monitor your llama server instance over a web ui.☆21Jun 16, 2026Updated last month
- Modular, agentic framework and MCP platform you self-host. Build and run your own AI agents, connect Claude Code, Claude Desktop, or Curs…☆20Updated this week
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Linux & Powershell scripts to easily set up and run the Qwen 3.5 series locally on Windows and Linux with llama.cpp.☆90Apr 28, 2026Updated 2 months ago
- ☆16Jun 13, 2026Updated last month
- Unified KV cache compression for LLM inference — TurboQuant, IsoQuant, PlanarQuant, TriAttention. 10 methods, GPU-validated, multi-GPU pl…☆24Jul 11, 2026Updated last week
- A markdown web renderer for AI agents — see the web without screenshots☆64May 21, 2026Updated 2 months ago
- ☆25Apr 28, 2023Updated 3 years ago
- llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 30…☆42Jul 2, 2026Updated 2 weeks ago
- A premium RAG-based AI Assistant built with React and FastAPI. Features efficient document indexing and high-accuracy retrieval-augmented…☆18Jun 3, 2026Updated last month