☆200Apr 5, 2026Updated 6 months ago
Alternatives and similar repositories for turboquant-model
Users that are interested in turboquant-model are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration☆1,814Sep 3, 2026Updated 2 months ago
- ♡☆23Aug 16, 2026Updated 2 months ago
- Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into VRAM☆17May 6, 2026Updated 5 months ago
- From-scratch PyTorch implementation of Google's TurboQuant (ICLR 2026) for LLM KV cache compression. 5x compression at 3-bit with 99.5% a…☆1,048Apr 23, 2026Updated 6 months ago
- # TurboQuant v3 (INT4 + AWQ + Protected Channels + Low-Rank) This notebook demonstrates a **TurboQuant-like** quantization algorithm: - G…☆17Mar 29, 2026Updated 7 months ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.☆405Apr 26, 2026Updated 6 months ago
- Unofficial implementation of HyperFlow for MiniMax H3 in ComfyUI. 8-step speedup with minimal quality loss☆18Oct 1, 2026Updated last month
- KV cache compression via block-diagonal rotation. Beats TurboQuant: better PPL (6.91 vs 7.07), 28% faster decode, 5.3x faster prefill, 44…☆1,058Apr 23, 2026Updated 6 months ago
- LLM inference in C/C++☆2,440Oct 22, 2026Updated last week
- Splish: a fork of Inco's Splash, tuned for Apple M5 GPUs. Apache-2.0☆30Oct 5, 2026Updated 3 weeks ago
- TurboQuant reference implementation — KV cache compression with engineering insights (ICLR 2026 paper reproduction)☆17Mar 28, 2026Updated 7 months ago
- This is a simple, local-only interface for interacting with the OpenAI API **You own KEY**, it can be run directly in your web browser. I…☆15Sep 3, 2025Updated last year
- Run any large transformer model on consumer GPUs. No quantization. No compromise.☆10Sep 23, 2026Updated last month
- The Screenplay Generator is a web application that allows users to generate a TV or movie screenplay based on a scene template.☆18Mar 18, 2023Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Code for the manuscript "A self-supervised multi-layer network of Rectified Spectral Units (ReSUs)" submitted to NeurIPS☆26Apr 6, 2026Updated 6 months ago
- interactive semantic search demo using Qwen3-0.6B-Embedding in your browser☆60Feb 25, 2026Updated 8 months ago
- Interested in running a conspiracy of agents (technical term) on your Local AI Infra? Who isn't!☆17Mar 30, 2026Updated 7 months ago
- Experimental Rust vector-compression sidecars implementing TurboQuant (ICLR 2026), PolarQuant and QJL research primitives, with exact rer…☆27Oct 25, 2026Updated last week
- An extension for oobabooga/text-generation-webui that automatically unloads and reloads your model.☆17Apr 22, 2024Updated 2 years ago
- Companion to Cracking the Forward Deployed AI Engineer Interview (2026 edition).☆14Aug 3, 2026Updated 3 months ago
- LLM inference in C/C++☆34Aug 1, 2026Updated 3 months ago
- LLM inference in C/C++☆57Oct 24, 2026Updated last week
- vLLM TurboQuant☆619Jun 25, 2026Updated 4 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Low latency, peer-to-peer, interactive game streaming.☆14Jan 22, 2024Updated 2 years ago
- Hermes Agent Hackathon: RealWorldTaskEnv — generate agentic training trajectories from real-world tasks for Hermes 4 fine-tuning☆16Mar 15, 2026Updated 7 months ago
- OpenMOSS pure C++ pipeline based on GGML☆75Sep 24, 2026Updated last month
- The benchmark for financial time series forecasting.☆17Jul 19, 2025Updated last year
- rvLLM for runpod serverless environment — lightweight, instant startup vLLM replacement☆40Apr 1, 2026Updated 7 months ago
- One-step Turkish text-to-speech trained with a drifting objective (Kyutai learned-temperature recipe)☆27Oct 16, 2026Updated 2 weeks ago
- Simulates the movement of stock price tick by tick.☆14Oct 4, 2025Updated last year
- A FastAPI-based TTS server that provides OpenAI-compatible API endpoints using KittenTTS☆17Aug 16, 2025Updated last year
- The official code of ALLECS: A Lightweight Language Error Correction System☆11Mar 12, 2023Updated 3 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- LLM inference in C/C++☆35Oct 15, 2026Updated 2 weeks ago
- Produce your own Dynamic 3.0 Quants and achieve optimum accuracy & SOTA quantization performance! Input a target size and the toolchain w…☆171Sep 11, 2026Updated last month
- Ultra-Sparse Adaptation of 1-Bit LLMs via XOR Patches☆90Aug 6, 2026Updated 2 months ago
- ☆26Mar 18, 2026Updated 7 months ago
- HIP/ROCm fork optimized for AMD RDNA2 (gfx1030) with PrismML Q1_0_G128 1-bit quant support, RotorQuant, TurboQuant, EAGLE3 and P-EAGLE sp…☆26Jun 23, 2026Updated 4 months ago
- The original Wine-NTsync userspace macOS prototype did not prove reliable enough for real-world use. Development therefore moved to WFUSy…☆16Aug 12, 2026Updated 2 months ago
- Turn anything into a CLI your agent can use. Securely.☆16Apr 20, 2026Updated 6 months ago