thu-ml / Adaptive-Sparse-TrainerLinks
Official implementation for "Pruning Large Language Models with Semi-Structural Adaptive Sparse Training" (AAAI 2025)
☆17Updated 5 months ago
Alternatives and similar repositories for Adaptive-Sparse-Trainer
Users that are interested in Adaptive-Sparse-Trainer are comparing it to the libraries listed below
Sorting:
- ☆18Updated last year
- KV cache compression via sparse coding☆17Updated 2 months ago
- [ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projection☆151Updated 10 months ago
- A sparse attention kernel supporting mix sparse patterns☆408Updated last week
- Official code for the paper "HEXA-MoE: Efficient and Heterogeneous-Aware MoE Acceleration with Zero Computation Redundancy"☆14Updated 9 months ago
- ☆11Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆159Updated 2 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆180Updated 3 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆360Updated 5 months ago
- [NeurIPS 2024 Oral🔥] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs.☆177Updated last year
- [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring☆261Updated 5 months ago
- [NeurIPS 2024] The official implementation of ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification☆31Updated 8 months ago
- The implementation for FREE-Merging: Fourier Transform for Model Merging with Lightweight Experts (ICCV25)☆10Updated 6 months ago
- Code for data-aware compression of DeepSeek models☆66Updated 2 weeks ago
- An algorithm for weight-activation quantization (W4A4, W4A8) of LLMs, supporting both static and dynamic quantization☆169Updated last month
- [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache☆345Updated last month
- [ICML 2025] Official PyTorch implementation of "FlatQuant: Flatness Matters for LLM Quantization"☆202Updated last month
- SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention☆199Updated this week
- [ICLR2025]: OSTQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitt…☆87Updated 8 months ago
- The Official Implementation of Ada-KV [NeurIPS 2025]☆120Updated last month
- Code Repository of Evaluating Quantized Large Language Models☆137Updated last year
- [CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>☆153Updated last month
- [NeurIPS'25] The official code implementation for paper "R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Tok…☆69Updated last week
- Efficient 2:4 sparse training algorithms and implementations☆58Updated last year
- SLiM: One-shot Quantized Sparse Plus Low-rank Approximation of LLMs (ICML 2025)☆32Updated last month
- [ICLR'25] ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation☆142Updated 9 months ago
- Code for Neurips24 paper: QuaRot, an end-to-end 4-bit inference of large language models.☆469Updated last year
- 16-fold memory access reduction with nearly no loss☆109Updated 9 months ago
- Code implementation of GPTAQ (https://arxiv.org/abs/2504.02692)☆78Updated 4 months ago
- Official Pytorch Implementation of "Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity"☆74Updated 5 months ago