Unified KV-cache compression for LLM inference: 12 Python-native methods, guarded add-on composition and routing, analytical capacity simulation, Godzilla KVarN/TriAttention with SM86/SM89 qualification, CUDA weight sharing, and multi-GPU planning.
☆30Oct 10, 2026Updated 3 weeks ago
Alternatives and similar repositories for multi-turboquant
Users that are interested in multi-turboquant are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into VRAM☆17May 6, 2026Updated 5 months ago
- Run Qwen3.5-35B-A3B with 1 million token context on a single NVIDIA L4☆29Apr 9, 2026Updated 6 months ago
- Open Source Probabilistic Networks Library from Intel☆11Apr 13, 2017Updated 9 years ago
- Object mapper for Riak and Redis☆13Mar 10, 2010Updated 16 years ago
- WMMA GEMM in ROCm for RDNA GPUs☆19Sep 1, 2026Updated last month
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆19Apr 25, 2026Updated 6 months ago
- This is a repository for documenting the setup and performance of MI100s in popular inference engines.☆20Oct 8, 2026Updated 3 weeks ago
- Fused MoE kernel☆23Oct 23, 2026Updated last week
- ☆41Mar 31, 2026Updated 7 months ago
- Ultra-low-latency, high-throughput multiprocess transport over SHM and mmap. LMAX-Disruptor-style cross-process ring substrate.☆22Aug 6, 2026Updated 2 months ago
- Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.☆20Aug 10, 2026Updated 2 months ago
- ☆27Jul 13, 2026Updated 3 months ago
- LLM inference in C/C++☆64May 7, 2026Updated 5 months ago
- antirez/ds4-style hybrid quant DeepSeek V4 Flash on a single DGX Spark via vLLM☆17May 11, 2026Updated 5 months ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- LLM inference in C/C++☆35Oct 15, 2026Updated 2 weeks ago
- Prototype of a high-performance C++ backtesting engine designed to analyse financial data and evaluate multiple trading strategies at sca…☆13Updated this week
- llama.cpp fork with TurboQuant quantization (turbo2/3/4) and TriAttention GPU-accelerated KV cache pruning. 75 tok/s on Qwen3-8B / RTX 30…☆57Jul 2, 2026Updated 3 months ago
- vLLM fork for Tesla V100 (SM70) — extends 1CatAI's AWQ support and adds GGUF support☆26Jun 20, 2026Updated 4 months ago
- The Fish Audio Infernence kernel opensource project.☆29Oct 15, 2026Updated 2 weeks ago
- ☆106Updated this week
- ☆43Updated this week
- LLM-agnostic memory layer for AI agents. No embeddings, no vector DB — just fast, structured, temporal memory that any LLM can consume as…☆11Apr 27, 2026Updated 6 months ago
- ☆13Oct 10, 2024Updated 2 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Control Android, Linux and Windows☆17Feb 26, 2016Updated 10 years ago
- Curated high-performance TIRx GPU kernels☆34Updated this week
- [ACL 2026] Official evaluation code for “MMErroR: A Benchmark for Erroneous Reasoning in Vision-Language Models”.☆20Apr 26, 2026Updated 6 months ago
- llama.cpp fork with TQ3_1S/4S CUDA kernels — 3.5-bit WHT quantization achieving Q4s quality at 10% smaller size. Based on RaBitQ-inspired…☆237Oct 13, 2026Updated 2 weeks ago
- DFlash vLLM for DGX Spark — Plug & Play Block-Diffusion Speculative Decoding☆53Oct 6, 2026Updated 3 weeks ago
- ☆18Oct 13, 2026Updated 2 weeks ago
- A Rust port of UVR.☆11Mar 14, 2024Updated 2 years ago
- GPU-initiated Large-scale GNN System [ATC 23]☆19Aug 31, 2026Updated 2 months ago
- Unlock WeMod Pro for free — bypass the 2-hour daily limit and unlock all Pro features. Windows WPF app, single-file 65 MB, fully reversib…☆42Oct 10, 2026Updated 3 weeks ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- FlashFunAsr: A lightweight vLLM implementation built from scratch for FunASR nano☆12Jan 5, 2026Updated 9 months ago
- Simulates the movement of stock price tick by tick.☆14Oct 4, 2025Updated last year
- Mixed-precision numerics benchmarks in Rust and Python - covering GEMMs, SYRKs, DOTs, and higher-level BLAS and LAPACK-style functionalit…☆33Updated this week
- A modular extension layer for the Caelestia desktop, with local OCR, region search, a local AI runtime, a Caelestia-styled SDDM login scr…☆26Updated this week
- Public repo for kernel writing skills in CuTeDSL, Triton, Tilelang and CUDA☆25Jul 7, 2026Updated 3 months ago
- a Matlab library of computational causal discovery and variable selection algorithms☆19Nov 25, 2019Updated 6 years ago
- Cutting Edge WebRTC Video Conferencing ported to .NET from https://github.com/versatica/mediasoup/☆12Apr 8, 2025Updated last year