vLLM TurboQuant
☆618Jun 25, 2026Updated 4 months ago
Alternatives and similar repositories for vllm-turboquant
Users that are interested in vllm-turboquant are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration☆1,809Sep 3, 2026Updated last month
- Claude Code native build of `autoresearch`: an autonomous experiment loop that writes session files, benchmarks changes, logs results, an…☆27Mar 13, 2026Updated 7 months ago
- Engineering Software Factory - open source, self-hosted, local AI first☆201Updated this week
- AI Coding Factory. Replace your software outsourcing vendor contracts with your own AI.☆183Feb 2, 2026Updated 8 months ago
- Aegis- a local zero-trust AI gate for OS and Apps packages☆31May 22, 2026Updated 5 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Welcome to Firecracker + AgentFS for AI Agents☆30May 18, 2026Updated 5 months ago
- KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44…☆1,057Apr 23, 2026Updated 6 months ago
- ☆22Apr 10, 2026Updated 6 months ago
- dede (DogEatDog.DependencyExplorer) is a static dependency and blast-radius explorer for mixed .NET workspaces with many sibling reposi…☆33Mar 10, 2026Updated 7 months ago
- ☆7,021Jul 20, 2026Updated 3 months ago
- Experimental llama.cpp fork for inference research and development☆893Updated this week
- An agent-first dependency and blast-radius explorer for Python codebases. Generates structured, machine-readable dependency graphs that A…☆19Mar 18, 2026Updated 7 months ago
- LLM inference in C/C++☆2,441Oct 22, 2026Updated last week
- DFlash vLLM for DGX Spark — Plug & Play Block-Diffusion Speculative Decoding☆53Oct 6, 2026Updated 3 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- TurboQuant+ KV cache compression for vLLM. 3.8x smaller KV cache, same conversation quality. Fused CUDA kernels with automatic PyTorch fa…☆82Sep 19, 2026Updated last month
- LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.☆405Apr 26, 2026Updated 6 months ago
- From-scratch PyTorch implementation of Google's TurboQuant (ICLR 2026) for LLM KV cache compression. 5x compression at 3-bit with 99.5% a…☆1,048Apr 23, 2026Updated 6 months ago
- RLM based security scanner for massive .NET codebases☆78Feb 9, 2026Updated 8 months ago
- FTE+AI: Vendor Replacement Program Framework☆81Dec 31, 2025Updated 9 months ago
- Adaptive Precision for EXpert Models: MoE-aware mixed-precision quantization☆482Oct 14, 2026Updated 2 weeks ago
- Fabric for Agents☆45Jul 20, 2026Updated 3 months ago
- ☆595Sep 5, 2026Updated last month
- Recursive Language Models Gateway build using rlm https://arxiv.org/abs/2512.24601v1☆127Jan 6, 2026Updated 9 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- LLM speculative inference server for heterogeneous hardware & consumer GPUs☆2,961Updated this week
- SkillOpt with local AI is a text-space optimizer that trains reusable natural-language skills for frozen LLM agents through trajectory-dr…☆86May 25, 2026Updated 5 months ago
- A vector index built on TurboQuant, written in Rust with Python bindings☆17,435Oct 7, 2026Updated 3 weeks ago
- DFlash: Block Diffusion for Flash Speculative Decoding☆6,208Aug 18, 2026Updated 2 months ago
- ☆397Apr 16, 2026Updated 6 months ago
- llama-benchy - llama-bench style benchmarking tool for all backends☆764Jul 10, 2026Updated 3 months ago
- Mixed-precision quantization for LLMs. Every layer refracts into a different format based on its sensitivity. Native compressed-tensors e…☆105Updated this week
- SimpleAgents lets anyone vibe-code LLM agents and ship them production-ready. It’s Rust-first with Python/Node/Go bindings, multi-provide…☆34Jun 6, 2026Updated 4 months ago
- ♡☆23Aug 16, 2026Updated 2 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆14Oct 5, 2026Updated 3 weeks ago
- Qwen3.6-35B-A3B-heretic NVFP4 + DFlash speculative decoding on DGX Spark (GB10/sm_121a). Source-built vLLM image + 7 patches + comprehens…☆147Oct 6, 2026Updated 3 weeks ago
- Background Coding Agents for Manufacturing Example Code☆23Dec 30, 2025Updated 10 months ago
- LMCache: Supercharge Your LLM with the Fastest KV Cache Layer☆12,224Updated this week
- Docker configuration for running VLLM on dual DGX Sparks☆2,457Updated this week
- ☆200Apr 5, 2026Updated 6 months ago
- Run AI models on Apple Silicon and Raspberry Pi☆46Updated this week