A high-throughput LLM serving engine with non-uniform KV cache compression, built on vLLM
☆16Sep 9, 2026Updated last month
Alternatives and similar repositories for tangram
Users that are interested in tangram are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- The official repository for QWHA☆13Sep 17, 2025Updated last year
- Accurate and fast KV cache compression with a gating mechanism☆33Jul 27, 2026Updated 3 months ago
- ☆14Mar 20, 2026Updated 7 months ago
- Algorithm-System Co-design: accurate and efficient 2-bit KV cache quantization for LLM Inference.☆23May 20, 2026Updated 5 months ago
- [ICLR2025] Code and data for paper: Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasonin…☆46Mar 10, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆22Oct 19, 2026Updated last week
- [NeurIPS 2023] Token-Scaled Logit Distillation for Ternary Weight Generative Language Models☆18Dec 6, 2023Updated 2 years ago
- A Client for Blue Dragon Club☆20Apr 11, 2025Updated last year
- An official PyTorch implementation of "Differentiable Learning of Generalized Structured Matrices for Efficient Deep Neural Networks," IC…☆11Apr 18, 2024Updated 2 years ago
- ☆39Mar 17, 2025Updated last year
- ☆11May 24, 2024Updated 2 years ago
- A novel variant of sliced Wasserstein based on a new slicing technique that utilizes the convolution operator.☆12Jan 14, 2023Updated 3 years ago
- Multi-GPU prefill acceleration for llama.cpp☆15Updated this week
- ☆30Sep 11, 2026Updated last month
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Official codebase for AdaRank: Adaptive Rank Pruning for Enhanced Model Merging (ICLR 2026)☆20Jan 26, 2026Updated 9 months ago
- Sparsity as a Variance Regulator for Improved Depth Utilization in Language Models☆16May 11, 2026Updated 5 months ago
- Code for the AAAI 2024 Oral paper "OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Model…☆74Mar 7, 2024Updated 2 years ago
- The evaluation framework for training-free sparse attention in LLMs☆134Jan 27, 2026Updated 9 months ago
- Qwen3.8-Flash-Next at full 262k context on 8 GB RAM and 16 GB VRAM—20 tok/s generation and 1,600 tok/s prefill.☆48Updated this week
- AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference☆21Jan 24, 2025Updated last year
- To mitigate position bias in LLMs, especially in long-context scenarios, we scale only one dimension of LLMs, reducing position bias and …☆13Jun 18, 2024Updated 2 years ago
- ☆12Sep 13, 2024Updated 2 years ago
- [ACL Findings 2026] Official Implementation of "FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acc…☆34Apr 14, 2026Updated 6 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Memora: A Long-Term Memory Benchmark for Personalized Agents☆19Jul 22, 2026Updated 3 months ago
- LLM Inference with Microscaling Format☆35Nov 12, 2024Updated last year
- "밑바닥부터 시작하는 데이터 사이언스" 예시 코드☆14Feb 5, 2020Updated 6 years ago
- HikVision IP Camera Alarm Server to MQTT☆14Sep 20, 2023Updated 3 years ago
- Korean Benchmark for Korean Legal Language Understanding☆19Aug 27, 2026Updated 2 months ago
- This repo contains the source code for: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs☆44Aug 14, 2024Updated 2 years ago
- Official implementation of LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents☆34Updated this week
- ☆37Oct 10, 2024Updated 2 years ago
- Official implementation of the paper: "A deeper look at depth pruning of LLMs"☆15Jul 24, 2024Updated 2 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Proxima lets existing GPUs serve 4x more concurrent requests☆26Aug 19, 2026Updated 2 months ago
- ☆14Oct 3, 2024Updated 2 years ago
- Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.☆13Jun 5, 2024Updated 2 years ago
- Multichannel software lock-in amplifier.☆15Updated this week
- ☆17Jul 25, 2023Updated 3 years ago
- Custom HIP inference engine for AMD MI50/MI60 (gfx906)☆18Oct 4, 2026Updated 3 weeks ago
- (CVPR'26) Gradient-free Layer Selection for Efficient Training of Deep Neural Networks☆12Mar 23, 2026Updated 7 months ago