KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.
☆508Jun 22, 2026Updated 3 months ago
Alternatives and similar repositories for KVarN
Users that are interested in KVarN are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Spire is a Python embedded domain-specific language (DSL) for RTL generation. Its built-in optimizations reduce area and delay of circuit…☆44Updated this week
- Welcome to the official repository of SINQ! A novel, fast and high-quality quantization method designed to make any Large Language Model …☆633May 8, 2026Updated 5 months ago
- [NeurIPS 2026 Spotlight] Fast, lossless LLM inference via dual-view diffusion decoding.☆482Aug 12, 2026Updated last month
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,151Updated this week
- LLM speculative inference server for heterogeneous hardware & consumer GPUs☆2,911Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆37Sep 23, 2026Updated 2 weeks ago
- 4-5x faster Qwen3.5 on ASUS GX10 / DGX Spark — Hybrid INT4+FP8 + MTP via one shell script☆32Apr 16, 2026Updated 5 months ago
- TokenSpeed is a speed-of-light LLM inference engine.☆2,211Updated this week
- DFlash: Block Diffusion for Flash Speculative Decoding☆6,148Aug 18, 2026Updated last month
- Efficient Decision tree Ensembles library for IoT edge nodes☆15Jan 29, 2025Updated last year
- DeepSeek V4 Flash specific inference engine. SSD MoE expert paging (slot-bank) + disk KV cache for long agent sessions. Metal-first, narr…☆32Sep 6, 2026Updated last month
- llama.cpp fork with additional SOTA quants and improved performance☆3,293Updated this week
- GPT-2-style LLM built from scratch in C/CUDA with hand-written backprop, BPE tokenizer, FlashAttention, pretraining, and SFT.☆123Jun 18, 2026Updated 3 months ago
- World's first Nintendo 3DS emulator for Apple devices based on Citra.☆18Apr 7, 2023Updated 3 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- DeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms☆7,204Jul 9, 2026Updated 3 months ago
- Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and m…☆2,375Updated this week
- Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables,…☆13,840Updated this week
- Experimental llama.cpp fork for inference research and development☆887Updated this week
- Run any local LLM engine, auto-tuned to your GPU — polished web UI + OpenAI/Anthropic-compatible API. Point Claude Code at your own machi…☆283Updated this week
- Messy is an open-source framework that integrates a RISC-V ISS with SystemC-AMS☆26Sep 8, 2025Updated last year
- Riscrithm is a lightweight macro-assembly compiler for RISC-V, providing readable control flow, modular files, compile-time validation, m…☆28Oct 2, 2026Updated last week
- The inference engine the open-source world built for itself.☆234Sep 27, 2026Updated 2 weeks ago
- LMCache: Supercharge Your LLM with the Fastest KV Cache Layer☆12,000Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Causal Parallel Tree Drafting☆232Sep 21, 2026Updated 2 weeks ago
- TurboQuant WASM SIMD vector compression — 3 bits/dim with fast dot product. Requires relaxed SIMD (Chrome 114+, Firefox 128+, Safari 18+,…☆323Apr 19, 2026Updated 5 months ago
- DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm☆23,769Updated this week
- Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails…☆7,084Updated this week
- KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44…☆1,054Apr 23, 2026Updated 5 months ago
- A database of knowledge around inference & training on GFX906 GPUs https://skyne98.github.io/wiki-gfx906/☆18Feb 21, 2026Updated 7 months ago
- A Plug-and-play Lightweight tool for the Inference Optimization of Deep Neural networks☆54May 8, 2026Updated 5 months ago
- Merkle tree implementation in Rust with configurable storage backends and hash functions. Fixed depth and incremental only. Optimized for…☆229Aug 12, 2026Updated last month
- A physics-grounded, cost-aware optimization loop for vLLM☆68Aug 22, 2026Updated last month
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Jacquard is a small programming language designed for a regime in which most code is written by machine-learning models and reviewed by p…☆120Updated this week
- Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM☆1,148Sep 24, 2026Updated 2 weeks ago
- Pure Rust Inference Engine☆704Updated this week
- PostgreSQL in-database durable execution☆2,832Updated this week
- ☆24Jun 16, 2026Updated 3 months ago
- Adaptive Test-time Learning and Autonomous Specialization☆2,103Updated this week
- LLM inference in C/C++☆2,426Updated this week