LLM inference in C/C++
☆64May 7, 2026Updated 5 months ago
Alternatives and similar repositories for llama-cpp-turboquant
Users that are interested in llama-cpp-turboquant are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44…☆1,054Apr 23, 2026Updated 6 months ago
- ☆41Mar 31, 2026Updated 6 months ago
- LLM inference in C/C++☆2,435Updated this week
- HIP/ROCm fork optimized for AMD RDNA2 (gfx1030) with PrismML Q1_0_G128 1-bit quant support, RotorQuant, TurboQuant, EAGLE3 and P-EAGLE sp…☆26Jun 23, 2026Updated 4 months ago
- llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 30…☆56Jul 2, 2026Updated 3 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity sim…☆28Oct 10, 2026Updated last week
- llama.cpp fork with TQ3_1S/4S CUDA kernels — 3.5-bit WHT quantization achieving Q4s quality at 10% smaller size. Based on RaBitQ-inspired…☆233Oct 13, 2026Updated last week
- TailCI is a lightweight, fast, and modern web application built with CodeIgniter and styled with Tailwind CSS. It combines the simplicity…☆16Oct 10, 2026Updated last week
- ☆21Aug 17, 2026Updated 2 months ago
- Distribute and run LLMs with a single file.☆25May 13, 2025Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs (Windows build & kernels)☆13Aug 15, 2026Updated 2 months ago
- Experimental llama.cpp fork for inference research and development☆888Updated this week
- SGLang for cool dudes using V100☆80Oct 5, 2026Updated 2 weeks ago
- Pure PyTorch implementation of HISA (Hierarchical Indexed Sparse Attention) as a drop-in plugin for HuggingFace models☆16Apr 7, 2026Updated 6 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,168Updated this week
- llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% t…☆424Oct 12, 2026Updated last week
- My blog site, now with Leptos 0.7, deployed to Fly.io☆14Apr 17, 2026Updated 6 months ago
- A hands-free voice assistant for your inbox and tools. Native iOS app with Gmail, Calendar, Slack, GitHub, Notion, Granola, and custom MC…☆29Updated this week
- Shadcn for icons.☆17Oct 3, 2026Updated 2 weeks ago
- Implementation for IceCache: Memory-Efficient KV-cache Management for Long-Sequence LLMs (ICLR 2026).☆25Jun 9, 2026Updated 4 months ago
- Flash Attention v2 for Pascal (P100, GTX 1080) and Volta (V100) GPUs — bringing O(N) memory attention to hardware the official flash-attn…☆27Apr 4, 2026Updated 6 months ago
- LLM inference in C/C++☆357Updated this week
- ☆21Jul 1, 2026Updated 3 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- LLM inference in C/C++☆972Updated this week
- Empower is an engine for visual no code programming, which aims to make complex programming intuitive and easy to understand for both adv…☆58Oct 5, 2026Updated 2 weeks ago
- ☆36Updated this week
- Let's try and finetune the OpenAI consistency decoder to work for SDXL☆25Dec 3, 2023Updated 2 years ago
- Omaxian is a port of Omarchy shell for Debian-based systems + X11/XLibre + i3. Same bar, themes, launcher, and omarchy-* commands — no Wa…☆20Updated this week
- relational database + coding = a dev superpower, so I wrote about what I’ve learned along the way …☆14Oct 10, 2025Updated last year
- Experimental LLM inference in C/C++☆37May 15, 2026Updated 5 months ago
- An OpenCode plugin that gives your agent access to DuckDuckGo, Brave, Google News, Wikipedia, Bluesky, MDN Web Docs, and the IETF, W3C an…☆19Sep 19, 2026Updated last month
- FlashInfer: Kernel Library for LLM Serving (Windows build & kernels)☆20Oct 13, 2026Updated last week
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Flexible OrcaSlicer for the Snapmaker U1 - Color Patch prints painted surfaces as a shell over a different-material core (soft grip on a …☆41Oct 16, 2026Updated last week
- vLLM fork for Tesla V100 (SM70) — extends 1CatAI's AWQ support and adds GGUF support☆26Jun 20, 2026Updated 4 months ago
- 𝗠𝗶𝗵𝗮𝗲𝗹𝗮 𝗞𝗢𝗦𝗘𝗩𝗔 - 𝗔𝗣𝗣 𝗟𝗜𝗘𝗗 𝗔𝗜/𝗠𝗟 · 𝗗𝗔𝗧𝗔 · 𝗦𝗪 𝗘𝗡𝗚𝗜𝗡𝗘𝗘𝗥. AI/ML Engineer focused on Artificial Intellige…☆36Oct 9, 2026Updated 2 weeks ago
- An analog synth in C# for Unity☆15Nov 3, 2017Updated 8 years ago
- A synthesizer/musical tool made in C# and running in Unity, with very beautiful chord transitions.☆10Jan 15, 2020Updated 6 years ago
- Task management app powered by React + Vite with Material UI, featuring advanced task handling, state management.☆19Feb 8, 2026Updated 8 months ago
- Adaptive Precision for EXpert Models: MoE-aware mixed-precision quantization☆481Oct 14, 2026Updated last week