thu-ml / Adaptive-Sparse-TrainerLinks
Official implementation for "Pruning Large Language Models with Semi-Structural Adaptive Sparse Training" (AAAI 2025)
☆18Updated 6 months ago
Alternatives and similar repositories for Adaptive-Sparse-Trainer
Users that are interested in Adaptive-Sparse-Trainer are comparing it to the libraries listed below
Sorting:
- ☆18Updated last year
- KV cache compression via sparse coding☆17Updated 3 months ago
- [NeurIPS 2024 Oral🔥] DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs.☆180Updated last year
- A sparse attention kernel supporting mix sparse patterns☆442Updated last week
- [ICLR'25] ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation☆146Updated 10 months ago
- The implementation for FREE-Merging: Fourier Transform for Model Merging with Lightweight Experts (ICCV25)☆13Updated 7 months ago
- Code for the paper “Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling”☆116Updated this week
- [ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring☆266Updated 6 months ago
- SLiM: One-shot Quantized Sparse Plus Low-rank Approximation of LLMs (ICML 2025)☆32Updated 2 months ago
- Github Repo for OATS: Outlier-Aware Pruning through Sparse and Low Rank Decomposition☆17Updated 9 months ago
- [ICLR 2025] Palu: Compressing KV-Cache with Low-Rank Projection☆153Updated 11 months ago
- [ICML 2025] SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models☆49Updated last year
- [ICLR2025]: OSTQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitt…☆87Updated 9 months ago
- An algorithm for weight-activation quantization (W4A4, W4A8) of LLMs, supporting both static and dynamic quantization☆172Updated 2 months ago
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs☆187Updated 4 months ago
- PyTorch implementation of PTQ4DiT https://arxiv.org/abs/2405.16005☆44Updated last year
- Code repository for ICLR 2025 paper "LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid"☆24Updated 10 months ago
- [NeurIPS'25] The official code implementation for paper "R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Tok…☆75Updated last week
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆368Updated 6 months ago
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆161Updated 3 months ago
- SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention☆258Updated last week
- 16-fold memory access reduction with nearly no loss☆109Updated 10 months ago
- Official Pytorch Implementation of "Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity"☆78Updated 6 months ago
- [COLM 2025] Official PyTorch implementation of "Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models"☆67Updated 6 months ago
- [ICML 2025] Official PyTorch implementation of "FlatQuant: Flatness Matters for LLM Quantization"☆207Updated 2 months ago
- [NeurIPS'25] dKV-Cache: The Cache for Diffusion Language Models☆129Updated 8 months ago
- The Official Implementation of Ada-KV [NeurIPS 2025]☆125Updated 2 months ago
- The code repository of "MBQ: Modality-Balanced Quantization for Large Vision-Language Models"☆75Updated 10 months ago
- [ACM MM2025]: MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization☆35Updated 5 months ago
- Official code for the paper "HEXA-MoE: Efficient and Heterogeneous-Aware MoE Acceleration with Zero Computation Redundancy"☆15Updated 10 months ago