Github Repo for OATS: Outlier-Aware Pruning through Sparse and Low Rank Decomposition
☆18Apr 16, 2025Updated 11 months ago
Alternatives and similar repositories for OATS
Users that are interested in OATS are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Official implementation for "Pruning Large Language Models with Semi-Structural Adaptive Sparse Training" (AAAI 2025)☆18Jul 1, 2025Updated 8 months ago
- Official Repo for SparseLLM: Global Pruning of LLMs (NeurIPS 2024)☆67Mar 27, 2025Updated 11 months ago
- Generic library for neural collapse and several derivative works on the phenomenon.☆18Apr 14, 2025Updated 11 months ago
- This repository contains code for the MicroAdam paper.☆21Dec 14, 2024Updated last year
- [ICLR 2025] Official implementation of paper "Dynamic Low-Rank Sparse Adaptation for Large Language Models".☆24Mar 16, 2025Updated last year
- SLiM: One-shot Quantized Sparse Plus Low-rank Approximation of LLMs (ICML 2025)☆35Nov 28, 2025Updated 3 months ago
- ☆30Jul 22, 2024Updated last year
- [ICML'24 Oral] APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference☆47Jun 4, 2024Updated last year
- ☆35May 24, 2024Updated last year
- Implementation of Effective Sparsification of Neural Networks with Global Sparsity Constraint☆31Mar 24, 2022Updated 4 years ago
- [ICML 2025] From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories and Applications☆52Oct 30, 2025Updated 4 months ago
- Lightweight, modular framework for training large models with a focus on scalability, clarity, and hackability.☆77Mar 17, 2026Updated last week
- ☆28Feb 21, 2025Updated last year
- Activation-aware Singular Value Decomposition for Compressing Large Language Models☆90Oct 22, 2024Updated last year
- Official implementation of the ICLR paper "Streamlining Redundant Layers to Compress Large Language Models"☆41May 1, 2025Updated 10 months ago
- Official Pytorch Implementation of Our Paper Accepted at ICLR 2024-- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLM…☆51Apr 9, 2024Updated last year
- [ICML 2024] Official Implementation of SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks☆39Feb 4, 2025Updated last year
- HALO: Hadamard-Assisted Low-Precision Optimization and Training method for finetuning LLMs. 🚀 The official implementation of https://arx…☆29Feb 17, 2025Updated last year
- Official repository for paper: O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning☆98Feb 21, 2025Updated last year
- Implementation of NM sparsity recipe presented in the paper "Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers".☆11Feb 5, 2024Updated 2 years ago
- ☆20Nov 26, 2025Updated 3 months ago
- [ICLR2026] The first W4A4KV4 quantized + 50% sparse LLMs!☆25Jan 26, 2026Updated last month
- ☆10Sep 2, 2023Updated 2 years ago
- ☆21Oct 2, 2024Updated last year
- ☆57Jun 10, 2024Updated last year
- Official Code of The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks[ICML2022]☆17Sep 20, 2022Updated 3 years ago
- Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid☆15Dec 6, 2022Updated 3 years ago
- udp并发实现代码，含udp server，udp client请求建立测试代码☆16Oct 26, 2024Updated last year
- [ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxia…☆29Jul 24, 2025Updated 8 months ago
- Pytorch implementation of our paper accepted by ICML 2023 -- "Bi-directional Masks for Efficient N:M Sparse Training"☆13Jun 7, 2023Updated 2 years ago
- ☆27Mar 29, 2025Updated 11 months ago
- Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity☆22Aug 28, 2025Updated 6 months ago
- Training with Block Minifloat number representation☆18May 2, 2021Updated 4 years ago
- [ICLRW'26] EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation☆29Mar 16, 2026Updated last week
- Design of High-Level Synthesis of Xilinx FFT IP core via FFT library☆14Jul 17, 2023Updated 2 years ago
- [ICLR 2025] The official pytorch implement of "Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Cont…☆72Sep 18, 2025Updated 6 months ago
- Official implementation for Text Generation Beyond Discrete Token Sampling☆24Aug 11, 2025Updated 7 months ago
- Awesome list for LLM pruning.☆290Oct 11, 2025Updated 5 months ago
- [ICML 2024] SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models☆21May 28, 2024Updated last year