TurboQuant+ KV cache compression for vLLM. 3.8x smaller KV cache, same conversation quality. Fused CUDA kernels with automatic PyTorch fallback.
☆80Jul 14, 2026Updated last month
Alternatives and similar repositories for turboquant-vllm
Users that are interested in turboquant-vllm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- llama.cpp fork with TQ3_1S/4S CUDA kernels — 3.5-bit WHT quantization achieving Q4s quality at 10% smaller size. Based on RaBitQ-inspired…☆232Updated this week
- forked from vllm-project/flash-attention☆64May 9, 2026Updated 4 months ago
- ☆28Mar 31, 2026Updated 5 months ago
- VENDORIZED in lucebox-hub. Fork of llama.cpp, ggml graph for lucebox inference engine☆31Jul 8, 2026Updated 2 months ago
- Deploy concurrent Hermes Agent workers on unified-memory GPUs (GB10, DGX Spark) for maximum total tok/s. Profile-isolated, kanban-coordin…☆80Jul 18, 2026Updated last month
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- NVIDIA Linux open GPU with P2P support☆511Sep 4, 2026Updated last week
- ☆17May 10, 2026Updated 4 months ago
- A simple GPT-3 interface to automate core legal writing tasks☆13Mar 8, 2023Updated 3 years ago
- Memory-bounded compressed sparse attention via streaming top-k. Triton kernels for the DeepSeek-V4 lightning indexer. 32x regime extensio…☆25May 5, 2026Updated 4 months ago
- FastAPI wrapper around original Vibevoice 1.5B and 7B models, with support for AWQ4 quant☆33Jun 22, 2026Updated 2 months ago
- The feed summarizer that powers feeds.carmo.io☆37Jan 1, 2026Updated 8 months ago
- pytorch+bert实现的意图识别与槽位填充☆11May 30, 2023Updated 3 years ago
- ☆62Feb 24, 2026Updated 6 months ago
- Developing a legal research tool leveraging ChatGPT / GPT-4☆14Mar 10, 2024Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- 基于langchain和chatglm6b构建的智能问答系统，支持自定义语料☆10Jun 25, 2023Updated 3 years ago
- Probe how GPT-n performs on statutory reasoning☆10Sep 17, 2024Updated last year
- vLLM TurboQuant☆617Jun 25, 2026Updated 2 months ago
- IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse☆141Mar 14, 2026Updated 5 months ago
- 爬取去哪网热门景点信息，抽取三元组信息，构建中文知识图谱☆13Apr 27, 2021Updated 5 years ago
- Chatbot_CN项目的知识图谱模块☆12Mar 27, 2020Updated 6 years ago
- Experimental llama.cpp fork for inference research and development☆834Updated this week
- llama.cpp fork with additional SOTA quants and improved performance☆3,217Updated this week
- Graph QABot Demo| 图谱问答案例☆14Apr 11, 2023Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- OpenCode local AI provider setup with automatic model syncing for LM Studio, vLLM, Ollama and any OpenAI-compatible API☆41Jul 10, 2026Updated 2 months ago
- An interactive course on prompt engineering for lawyers built using streamlit.☆19Sep 1, 2024Updated 2 years ago
- ☆17Aug 13, 2026Updated last month
- LLM inference in C/C++☆35Aug 24, 2026Updated 2 weeks ago
- Replayable Browser Agent☆16Apr 24, 2026Updated 4 months ago
- "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files☆11Oct 27, 2024Updated last year
- everything i know about cuda and triton☆13Jan 28, 2025Updated last year
- Enhancing the convergence speed by 2x and improving the training success of Physics-Informed Neural Networks (PINNs).☆13Oct 14, 2024Updated last year
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆973Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- 中国法研杯 CAIL 2019☆13Jun 17, 2019Updated 7 years ago
- 基于舆情中文核心论文的deepsearch项目☆16Apr 1, 2025Updated last year
- ☆19Dec 9, 2023Updated 2 years ago
- bf16 LoRA fine-tuning of [Qwen3.5-35B-A3B](https://huggingface.co/unsloth/Qwen3.5-35B-A3B) (a 35B-total / 3B-active Mixture-of-Experts vi…☆18Mar 12, 2026Updated 6 months ago
- 基于ASP.NET的图书租赁管理系统 ，运行操作请查看ReadMe.txt文件☆11Apr 20, 2016Updated 10 years ago
- A small, crazy fast hybrid search engine written in Rust.☆30Jun 27, 2026Updated 2 months ago
- a fast and lightweight distributed background task processing framework with seamless scheduling.☆15Mar 30, 2026Updated 5 months ago