TurboQuant+ KV cache compression for vLLM. 3.8x smaller KV cache, same conversation quality. Fused CUDA kernels with automatic PyTorch fallback.
☆74Jul 14, 2026Updated this week
Alternatives and similar repositories for turboquant-vllm
Users that are interested in turboquant-vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- llama.cpp fork with TQ3_1S/4S CUDA kernels — 3.5-bit WHT quantization achieving Q4s quality at 10% smaller size. Based on RaBitQ-inspired…☆219Jul 6, 2026Updated last week
- ☆60Feb 24, 2026Updated 4 months ago
- Deploy concurrent Hermes Agent workers on unified-memory GPUs (GB10, DGX Spark) for maximum total tok/s. Profile-isolated, kanban-coordin…☆71Updated this week
- ☆42Nov 13, 2025Updated 8 months ago
- ☆15May 10, 2026Updated 2 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A simple GPT-3 interface to automate core legal writing tasks☆13Mar 8, 2023Updated 3 years ago
- ☆27Nov 3, 2025Updated 8 months ago
- ☆23Dec 7, 2023Updated 2 years ago
- ROCmFPX Family for AMD Hardware and Processors. More quants and special agent quants☆107Updated this week
- Memory-bounded compressed sparse attention via streaming top-k. Triton kernels for the DeepSeek-V4 lightning indexer. 32x regime extensio…☆22May 5, 2026Updated 2 months ago
- Chief - the easiest way to Ralph☆34Jan 19, 2026Updated 5 months ago
- pytorch+bert实现的意图识别与槽位填充☆11May 30, 2023Updated 3 years ago
- vLLM TurboQuant☆610Jun 25, 2026Updated 2 weeks ago
- Developing a legal research tool leveraging ChatGPT / GPT-4☆14Mar 10, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A zero-latency, local-first runtime firewall for LLMs. Intercept every prompt and response locally to stop data leaks and runaway token c…☆19Jul 5, 2026Updated last week
- Row-wise block scaling for fp8 quantization matrix multiplication. Solution to GPU mode AMD challenge.☆19Feb 9, 2026Updated 5 months ago
- LLM inference in C/C++☆36Apr 12, 2026Updated 3 months ago
- ☆19Aug 19, 2025Updated 10 months ago
- Deterministic security layer for Openclaw(Clawdbot), Cursor and Claude Code. Write secure code, prevent data exfil, and more☆44Feb 5, 2026Updated 5 months ago
- Probe how GPT-n performs on statutory reasoning☆10Sep 17, 2024Updated last year
- Android Bluetooth gamepad project sample code.☆14Aug 17, 2022Updated 3 years ago
- Deploying full-stack on-prem deep research agent that can be run entirely on a local machine for $0!☆34Nov 8, 2025Updated 8 months ago
- LLAMA Turboquant implementation with CUDA support☆687Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆72Updated this week
- AI医生☆10May 27, 2020Updated 6 years ago
- 本项目主要研究大模型在单独的法律数据集上的效果，现在支持belle和chatglm相关的模型训练，预测，验证和在线部署， 另外增加爬虫代码，langchain，结合数据库预测等功能。☆12Jul 16, 2023Updated 2 years ago
- 爬取去哪网热门景点信息，抽取三元组信息，构建中文知识图谱☆13Apr 27, 2021Updated 5 years ago
- IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse☆124Mar 14, 2026Updated 4 months ago
- Nex Venus Communication Library☆75Nov 17, 2025Updated 7 months ago
- Simple intermediate representation language for learning and research.☆22Mar 27, 2020Updated 6 years ago
- Browse the Hytale server code using Patcher☆50Updated this week
- ☆24May 26, 2026Updated last month
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆20Oct 25, 2025Updated 8 months ago
- Experiments codes for COLING '22 paper "Augmenting Legal Judgment Prediction with Contrastive Case Relations"☆11Apr 25, 2024Updated 2 years ago
- An interactive course on prompt engineering for lawyers built using streamlit.☆19Sep 1, 2024Updated last year
- Utility programs to pipe data across a RDMA-capable network☆20Mar 14, 2026Updated 4 months ago
- ☆16Jun 15, 2026Updated 3 weeks ago
- A collection of GPU experiments and benchmarks for my personal understanding and research.☆33Updated this week
- A PDF unredactor.☆42Feb 7, 2026Updated 5 months ago