A high-throughput LLM serving engine with non-uniform KV cache compression, built on vLLM
☆16Nov 9, 2026Updated last week
Alternatives and similar repositories for tangram
Users that are interested in tangram are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Accurate and fast KV cache compression with a gating mechanism☆35Jul 27, 2026Updated 3 months ago
- [NeurIPS 2023] Token-Scaled Logit Distillation for Ternary Weight Generative Language Models☆18Dec 6, 2023Updated 2 years ago
- The official repository for QWHA☆13Sep 17, 2025Updated last year
- ☆14Mar 20, 2026Updated 7 months ago
- ☆10Nov 4, 2022Updated 4 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- [ICLR2025] Code and data for paper: Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasonin…☆46Mar 10, 2025Updated last year
- TernGEMM: General Matrix Multiply Library with Ternary Weights for Fast DNN Inference☆14Feb 22, 2022Updated 4 years ago
- A text-based adventure loosely based on the popular murder mystery game "Clue."☆10Oct 10, 2018Updated 8 years ago
- Algorithm-System Co-design: accurate and efficient 2-bit KV cache quantization for LLM Inference.☆25May 20, 2026Updated 5 months ago
- ☆24Oct 19, 2026Updated 3 weeks ago
- A Client for Blue Dragon Club☆20Apr 11, 2025Updated last year
- An official PyTorch implementation of "Differentiable Learning of Generalized Structured Matrices for Efficient Deep Neural Networks," IC…☆11Apr 18, 2024Updated 2 years ago
- ☆39Mar 17, 2025Updated last year
- Human Pose Estimation Papers ,Projects and Other Resource.☆10Dec 9, 2018Updated 7 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆11May 24, 2024Updated 2 years ago
- Layer-wise Pruning of Transformer Heads for Efficient Language Modeling☆22Feb 22, 2022Updated 4 years ago
- ☆31Sep 11, 2026Updated 2 months ago
- Official codebase for AdaRank: Adaptive Rank Pruning for Enhanced Model Merging (ICLR 2026)☆20Jan 26, 2026Updated 9 months ago
- A novel variant of sliced Wasserstein based on a new slicing technique that utilizes the convolution operator.☆12Jan 14, 2023Updated 3 years ago
- The evaluation framework for training-free sparse attention in LLMs☆134Jan 27, 2026Updated 9 months ago
- [ACL Findings 2026] Official Implementation of "FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acc…☆34Apr 14, 2026Updated 7 months ago
- ☆12Sep 13, 2024Updated 2 years ago
- AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference☆21Jan 24, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A sparse-first inference engine (SparseEngine).☆110Nov 11, 2026Updated last week
- Code for the AAAI 2024 Oral paper "OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Model…☆85Mar 7, 2024Updated 2 years ago
- LLM Inference with Microscaling Format☆35Nov 12, 2024Updated 2 years ago
- "밑바닥부터 시작하는 데이터 사이언스" 예시 코드☆14Feb 5, 2020Updated 6 years ago
- Sparsity as a Variance Regulator for Improved Depth Utilization in Language Models☆16May 11, 2026Updated 6 months ago
- Korean Benchmark for Korean Legal Language Understanding☆19Aug 27, 2026Updated 2 months ago
- This repo contains the source code for: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs☆44Aug 14, 2024Updated 2 years ago
- Port of Facebook's LLaMA model in C/C++☆21Nov 6, 2023Updated 3 years ago
- Official implementation of the paper: "A deeper look at depth pruning of LLMs"☆15Jul 24, 2024Updated 2 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆14Oct 3, 2024Updated 2 years ago
- Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.☆13Jun 5, 2024Updated 2 years ago
- ☆37Oct 10, 2024Updated 2 years ago
- To mitigate position bias in LLMs, especially in long-context scenarios, we scale only one dimension of LLMs, reducing position bias and …☆13Jun 18, 2024Updated 2 years ago
- HikVision IP Camera Alarm Server to MQTT☆14Sep 20, 2023Updated 3 years ago
- Official implementation of LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents☆34Oct 23, 2026Updated 3 weeks ago
- ☆17Jul 25, 2023Updated 3 years ago
Are these results useful?