☆200Apr 5, 2026Updated 6 months ago
Alternatives and similar repositories for turboquant-model
Users that are interested in turboquant-model are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration☆1,797Sep 3, 2026Updated last month
- Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into VRAM☆17May 6, 2026Updated 5 months ago
- From-scratch PyTorch implementation of Google's TurboQuant (ICLR 2026) for LLM KV cache compression. 5x compression at 3-bit with 99.5% a…☆1,048Apr 23, 2026Updated 5 months ago
- LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.☆403Apr 26, 2026Updated 5 months ago
- KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44…☆1,054Apr 23, 2026Updated 5 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆41Mar 31, 2026Updated 6 months ago
- LLM inference in C/C++☆2,426Updated this week
- The Screenplay Generator is a web application that allows users to generate a TV or movie screenplay based on a scene template.☆18Mar 18, 2023Updated 3 years ago
- Code for the manuscript "A self-supervised multi-layer network of Rectified Spectral Units (ReSUs)" submitted to NeurIPS☆26Apr 6, 2026Updated 6 months ago
- Ultra-Sparse Adaptation of 1-Bit LLMs via XOR Patches☆91Aug 6, 2026Updated 2 months ago
- OpenMOSS pure C++ pipeline based on GGML☆73Sep 24, 2026Updated 2 weeks ago
- interactive semantic search demo using Qwen3-0.6B-Embedding in your browser☆60Feb 25, 2026Updated 7 months ago
- vLLM TurboQuant☆617Jun 25, 2026Updated 3 months ago
- Experimental Rust vector-compression sidecars implementing TurboQuant (ICLR 2026), PolarQuant and QJL research primitives, with exact rer…☆26Sep 30, 2026Updated 2 weeks ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- LLM inference in C/C++☆57Updated this week
- Displays A Custom Profile Effect And Avatar Decoration.☆17Jul 11, 2024Updated 2 years ago
- HIP/ROCm fork optimized for AMD RDNA2 (gfx1030) with PrismML Q1_0_G128 1-bit quant support, RotorQuant, TurboQuant, EAGLE3 and P-EAGLE sp…☆26Jun 23, 2026Updated 3 months ago
- rvLLM for runpod serverless environment — lightweight, instant startup vLLM replacement☆40Apr 1, 2026Updated 6 months ago
- Provides the screen image for multimodal models when you send a message.☆13Nov 1, 2025Updated 11 months ago
- High-performance batched Top-K selection for CPU inference. Up to 80x faster than PyTorch, optimized for LLM sampling with AVX2 SIMD.☆18Mar 20, 2026Updated 6 months ago
- Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity sim…☆29Updated this week
- Production Claude Code skills for n8n from a Verified Creator's 100+ workflows☆32Apr 26, 2026Updated 5 months ago
- Code Mode inspired local sandboxed MCP Gateway - collapses N servers x M tools into 2 tools (~1,000 tokens)☆153Updated this week
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- ☆180Aug 10, 2025Updated last year
- Produce your own Dynamic 3.0 Quants and achieve optimum accuracy & SOTA quantization performance! Input a target size and the toolchain w…☆170Sep 11, 2026Updated last month
- 台湾大学李宏毅教授课程作业练习☆10Sep 2, 2021Updated 5 years ago
- AutoGPT maintainer/reviewer system☆16May 26, 2023Updated 3 years ago
- ☆17May 16, 2025Updated last year
- Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware☆523Sep 15, 2026Updated 3 weeks ago
- MVU Zod Status Menu Builder☆17May 19, 2026Updated 4 months ago
- A POC for a mutual fund listing app with a user signup/signin flow that displays multiple mutual funds in a scrollable list. Certain esse…☆10Feb 1, 2023Updated 3 years ago
- A skill that convenes a panel of CLI-based AI agents (Claude Code, Codex, Gemini CLI) to deliberate on engineering problems through struc…☆91Apr 7, 2026Updated 6 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- AI-powered Git CLI tool for smart commits, branches, and PRs☆33Nov 29, 2025Updated 10 months ago
- OpenUI Generative UI tool for Open WebUI - renders interactive charts, forms, tables, and cards in chat via OpenUI Lang.☆69May 25, 2026Updated 4 months ago
- TurboQuant WASM SIMD vector compression — 3 bits/dim with fast dot product. Requires relaxed SIMD (Chrome 114+, Firefox 128+, Safari 18+,…☆323Apr 19, 2026Updated 5 months ago
- ☆16Feb 3, 2026Updated 8 months ago
- Just an UI for Chatterbox, which uses about 1-2 GB RAM. Double click and you're good to go.☆24Jun 11, 2026Updated 4 months ago
- Steerable Harnesses for DeepAgents — Orchestra🪶☆17Sep 29, 2026Updated 2 weeks ago
- ☆27Sep 25, 2024Updated 2 years ago