KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.
☆513Jun 22, 2026Updated 4 months ago
Alternatives and similar repositories for KVarN
Users that are interested in KVarN are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Pythonic interface and JIT compiler for https://gitcode.com/cann/pto-isa☆28Jun 1, 2026Updated 5 months ago
- Code for the Paper GENIAL: Generative Design Space Exploration via Network Inversion for Low Power Algorithmic Logic Units☆25May 22, 2026Updated 5 months ago
- Welcome to the official repository of AC-LORA: (Almost) Training-Free Access Control-Aware Multi-Modal LLMs, a mechanism that provides tr…☆21Nov 14, 2025Updated 11 months ago
- ☆18Sep 26, 2025Updated last year
- Custom kernel collections using https://github.com/PTO-ISA/pto-isa☆23Updated this week
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Spire is a Python embedded domain-specific language (DSL) for RTL generation. Its built-in optimizations reduce area and delay of circuit…☆51Oct 13, 2026Updated 2 weeks ago
- Welcome to the official repository of SINQ! A novel, fast and high-quality quantization method designed to make any Large Language Model …☆636May 8, 2026Updated 5 months ago
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,179Updated this week
- Autonomous self-improving 4x DGX Spark (GB10) MoA stack + LoRA loop (DSV4F router, Qwen3.6/Omni/TwoTower/Gemma). Hermes MoA routing, ~90%…☆24Aug 17, 2026Updated 2 months ago
- DSpark-style speculative decoding draft head for Qwen3.6-27B AEON: training recipe, paired eval harness, vLLM patches. +11% serve tok/s v…☆15Jul 2, 2026Updated 4 months ago
- Megakernel for Gated DeltaNet, optimized by PTO-ISA☆23Oct 19, 2026Updated 2 weeks ago
- Universal AI computer-control KVM (REST API + MCP) that drives any machine as plain USB hardware — nothing installed on the target. Built…☆21Oct 6, 2026Updated 3 weeks ago
- [NeurIPS 2026 Spotlight] Fast, lossless LLM inference via dual-view diffusion decoding.☆483Aug 12, 2026Updated 2 months ago
- RTLScout is a agentic RTL optimization framework☆67Oct 26, 2026Updated last week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- LLM speculative inference server for heterogeneous hardware & consumer GPUs☆2,970Updated this week
- Linux anonymity / OPSEC toolkit — Tor-by-default with a kill switch, leak verification, and pluggable upstreams. Authorized use only.☆24Sep 2, 2026Updated 2 months ago
- Recover usable memory on DGX Spark with a 64 KiB kernel: an opt-in NVIDIA UVM page-table packing patch, build tools, and measured validat…☆35Oct 16, 2026Updated 2 weeks ago
- ☆41Aug 16, 2026Updated 2 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆3,364Updated this week
- DeepSeek V4 Flash @ 1M token context on 2x NVIDIA DGX Spark — production-tested recipe (45 tok/s decode, real 800K prompts served)☆37Jul 13, 2026Updated 3 months ago
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,783Updated this week
- TokenSpeed is a speed-of-light LLM inference engine.☆2,841Updated this week
- Headless / multi-user serving-mode for the DGX Spark: free unified memory for vLLM by paring back the desktop. Companion to qwen3.5-122B-…☆21Jun 29, 2026Updated 4 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- DFlash: Block Diffusion for Flash Speculative Decoding☆6,210Aug 18, 2026Updated 2 months ago
- Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and m…☆2,468Updated this week
- GLM-5.3-Flash (Z.ai) NVFP4 (LibertAI) + NVFP4 KV, 1M context on 2x DGX Spark GB10 — vLLM TP2 recipe, walls ledger, benchmarks, needle to …☆29Aug 31, 2026Updated 2 months ago
- GLM-5.2 Quantrio INT4/INT8 Mixed Abliterated — SPEED=1 C1≈30 tok/s @ 128k on 4x DGX Spark. Step-by-step recipe, image bake, results.☆13Jul 18, 2026Updated 3 months ago
- LLM inference in C/C++☆34Aug 1, 2026Updated 3 months ago
- GLM-5.2 QuantTrio TP=4+DCP2 on 4x NVIDIA DGX Spark (GB10) — tuned and measured under a real agent workload. Adaptive MTP, the indexer law…☆19Aug 24, 2026Updated 2 months ago
- GLM-5.2-Int4-Int8 on 8x GB10 cluster☆39Jul 30, 2026Updated 3 months ago
- Language model tokenization at GB/s☆4,150Updated this week
- MiniMax-H3 on single DGX Spark: keys Sol-Engine ports + Heretic TE + Spectrum/Sage speed + 0.5MP upscaler finish. Formal bench included.☆49Aug 23, 2026Updated 2 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- A Circuit Design Frontend based on Python☆26Sep 20, 2026Updated last month
- MiMo-V2.5 NVFP4 4-bit weights + NVFP4 4-bit KV cache + DFlash speculative decoding on 2× NVIDIA DGX Spark — 1M context, 3.4M-token KV poo…☆18Jul 13, 2026Updated 3 months ago
- Run the AEON Bench suite on your own hardware: verified HuggingFace pull → serve → benchmark (text · agentic ×3 harnesses · vision · audi…☆26Oct 6, 2026Updated 3 weeks ago
- DeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms☆7,277Jul 9, 2026Updated 3 months ago
- a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-…☆475Updated this week
- Implementation for IceCache: Memory-Efficient KV-cache Management for Long-Sequence LLMs (ICLR 2026).☆25Jun 9, 2026Updated 4 months ago
- 4-5x faster Qwen3.5 on ASUS GX10 / DGX Spark — Hybrid INT4+FP8 + MTP via one shell script☆32Apr 16, 2026Updated 6 months ago