☆46Oct 11, 2025Updated last year
Alternatives and similar repositories for DiffKV
Users that are interested in DiffKV are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆25Mar 20, 2021Updated 5 years ago
- [COLM 2024] SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models☆25Oct 5, 2024Updated 2 years ago
- Repo for ECHO: Efficient KV Cache Offloading with Lossless Prefetching for Serving Native Sparse Attention LLMs (OSDI'26)☆41Jul 12, 2026Updated 3 months ago
- ☆47Nov 25, 2024Updated last year
- code for paper Accurate KV Cache Quantization with Outlier Tokens Tracing (ACL2025 Main)☆11Jul 4, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆17Jun 4, 2026Updated 4 months ago
- ☆208Jul 15, 2025Updated last year
- Artifact for "Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving" [SOSP '24]☆25Nov 21, 2024Updated last year
- ☆18Sep 27, 2026Updated last month
- A graph linear algebra overlay☆52Apr 26, 2023Updated 3 years ago
- ☆21Jun 9, 2025Updated last year
- SC'25 UltraAttn: Efficiently Parallelizing Attention through Hierarchical Context-Tiling☆16Aug 14, 2025Updated last year
- ☆27Oct 1, 2025Updated last year
- Tutti is a GPU-centric KV Cache Store to make SSD work as DRAM.☆69Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- [NeurIPS'25 Spotlight] Adaptive Attention Sparsity with Hierarchical Top-p Pruning☆110Aug 31, 2026Updated 2 months ago
- Dynamic Context Selection for Efficient Long-Context LLMs☆65May 20, 2025Updated last year
- Layered prefill changes the scheduling axis from tokens to layers and removes redundant MoE weight reloads while keeping decode stall fre…☆24Mar 9, 2026Updated 7 months ago
- a simple API to use CUPTI☆10Aug 19, 2025Updated last year
- [ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference☆316May 1, 2025Updated last year
- High-Throughput, Cost-Effective Billion-Scale Vector Search with a Single GPU [SIGMOD'26]☆32Oct 20, 2026Updated last week
- A Generic Distributed Auto-Tuning Infrastructure☆24Jul 29, 2021Updated 5 years ago
- The repository for ATC'25 paper "Greyhound: Hunting Fail-Slows in Hybrid-Parallel Training at Scale"☆17Sep 7, 2026Updated last month
- ☆16Jun 4, 2024Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- GARNET: Reduced-Rank Topology Learning for Robust and Scalable Graph Neural Networks☆36Oct 1, 2023Updated 3 years ago
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)☆195Jul 10, 2024Updated 2 years ago
- The official code implementation for paper "PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs"☆33May 24, 2025Updated last year
- Artifact Evaluation for SOSP 2025☆23Aug 16, 2025Updated last year
- [USENIX ATC'25] GeneralSparse: Bridging the Gap in SpMM for Pruned Large Language Model Inference on GPUs☆14Feb 4, 2026Updated 8 months ago
- Code repository for the SOSP'25 paper DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism.☆23Nov 28, 2025Updated 11 months ago
- TA's implementation for the project of Computer Architecture and Intelligent Chip Design (23 Spring)☆10May 20, 2023Updated 3 years ago
- Artifact for OSDI'23: MGG: Accelerating Graph Neural Networks with Fine-grained intra-kernel Communication-Computation Pipelining on Mult…☆40Mar 17, 2024Updated 2 years ago
- [MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Se…☆867Mar 6, 2025Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆19Apr 15, 2025Updated last year
- A simple Mali 6xx/7xx register interface model that doesn't do any rendering.☆13Jan 29, 2016Updated 10 years ago
- flash attention 优化日志☆34Aug 28, 2026Updated 2 months ago
- [NeurIPS 2024] The official implementation of ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification☆33Mar 30, 2025Updated last year
- 实现和扩展RISC-V SBI运行时，使之能够支持并运行操作系统☆14Apr 1, 2025Updated last year
- ☆19Jun 3, 2023Updated 3 years ago
- ☆16Jul 19, 2026Updated 3 months ago