TurboQuant KV cache compression plugin for vLLM — asymmetric K/V, 8 models validated, consumer GPUs
☆51Apr 10, 2026Updated 6 months ago
Alternatives and similar repositories for turboquant-vllm
Users that are interested in turboquant-vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- vLLM TurboQuant☆618Jun 25, 2026Updated 3 months ago
- A simple GPT-3 interface to automate core legal writing tasks☆13Mar 8, 2023Updated 3 years ago
- pytorch+bert实现的意图 识别与槽位填充☆11May 30, 2023Updated 3 years ago
- Developing a legal research tool leveraging ChatGPT / GPT-4☆14Mar 10, 2024Updated 2 years ago
- 基于langchain和chatglm6b构建的智能问答系统，支持自定义语料☆10Jun 25, 2023Updated 3 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Nano vLLM☆13Jun 26, 2025Updated last year
- Probe how GPT-n performs on statutory reasoning☆10Sep 17, 2024Updated 2 years ago
- First open-source KVTC implementation (NVIDIA, ICLR 2026) -- 8-32x KV cache compression via PCA + adaptive quantization + entropy coding☆23Apr 17, 2026Updated 5 months ago
- A question answering AI tool for the content from the PDF files of the Civil Code, Criminal Code, Code of Criminal Procedure, Labor Stand…☆12May 14, 2023Updated 3 years ago
- AI-assisted software development methodology for solo builders☆30Updated this week
- ☆18Mar 31, 2026Updated 6 months ago
- AI医生☆10May 27, 2020Updated 6 years ago
- A MCP (Model Context Protocol) server to allow code security reviews using https://osv.dev (Open Source Vulnerabilities Database)☆15Dec 24, 2025Updated 9 months ago
- 基于pytorch_rnn的古诗词生成☆12Oct 24, 2021Updated 4 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆11Nov 13, 2020Updated 5 years ago
- ☆10Aug 28, 2018Updated 8 years ago
- Graph QABot Demo| 图谱问答案例☆14Apr 11, 2023Updated 3 years ago
- ☆13Oct 10, 2024Updated 2 years ago
- ☆12Apr 17, 2025Updated last year
- Experiments codes for COLING '22 paper "Augmenting Legal Judgment Prediction with Contrastive Case Relations"☆11Apr 25, 2024Updated 2 years ago
- Pure Rust OCR pipeline that re-implements the PaddleOCR (DBNet + SVTR\_HGNet) models without relying on C/C++ runtimes.☆18Updated this week
- AutoMapper website☆14Aug 19, 2020Updated 6 years ago
- Evaluation of BEIR Datasets using ColBERT retrieval model☆18Mar 4, 2022Updated 4 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Optimizing Deep Convolutional Neural Network with Ternarized Weights and High Accuracy☆16Jan 27, 2019Updated 7 years ago
- An integration of Qdrant ANN vector database backend with txtai☆25Jun 23, 2026Updated 3 months ago
- 中国法研杯 CAIL 2019☆13Jun 17, 2019Updated 7 years ago
- 基于舆情中文核心论文的deepsearch项目☆16Apr 1, 2025Updated last year
- 基于ASP.NET的图书租赁管理系统 ，运行操作请查看ReadMe.txt文件☆11Apr 20, 2016Updated 10 years ago
- Mixed-precision quantization for LLMs. Every layer refracts into a different format based on its sensitivity. Native compressed-tensors e…☆103Updated this week
- Unit Scaling demo and experimentation code☆15Mar 12, 2024Updated 2 years ago
- 医疗系统知识图谱问答☆12Feb 3, 2022Updated 4 years ago
- Python tool for exploiting CVE-2021-35616☆11Dec 3, 2021Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Legal Juegment Prediction (LJP) with BiLSTM and Attention☆13Jan 10, 2019Updated 7 years ago
- Alex Graves' Adaptive Computation Time in PyTorch☆14Jan 9, 2018Updated 8 years ago
- [ICML 2021] "Double-Win Quant: Aggressively Winning Robustness of Quantized DeepNeural Networks via Random Precision Training and Inferen…☆16Feb 13, 2022Updated 4 years ago
- Using SeqJudge to solve Legal Judgment Prediction.☆16Aug 24, 2018Updated 8 years ago
- CSS3与页面布局学习总结（一）——概要、选择器、特殊性与刻度单位☆14Nov 24, 2016Updated 9 years ago
- Samples on how to use filters to authenticate ASP .Net Web Api 2☆10Oct 4, 2016Updated 10 years ago
- A lightweight OAuth Integration Proxy.☆14Jan 8, 2023Updated 3 years ago