horseee / LLaMA-Pruning
Structural Pruning for LLaMA
☆54Updated last year
Alternatives and similar repositories for LLaMA-Pruning:
Users that are interested in LLaMA-Pruning are comparing it to the libraries listed below
- QuIP quantization☆52Updated last year
- Activation-aware Singular Value Decomposition for Compressing Large Language Models☆59Updated 5 months ago
- Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs☆80Updated 4 months ago
- PB-LLM: Partially Binarized Large Language Models☆152Updated last year
- Unofficial implementations of block/layer-wise pruning methods for LLMs.☆66Updated 11 months ago
- Reorder-based post-training quantization for large language model☆186Updated last year
- Work in progress.☆50Updated 2 weeks ago
- Low-bit optimizers for PyTorch☆125Updated last year
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆40Updated last year
- Advanced Ultra-Low Bitrate Compression Techniques for the LLaMA Family of LLMs☆111Updated last year
- ☆145Updated last year
- Repository for CPU Kernel Generation for LLM Inference☆25Updated last year
- Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"☆27Updated last year
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆47Updated last year
- [ICLR 2024] Jaiswal, A., Gan, Z., Du, X., Zhang, B., Wang, Z., & Yang, Y. Compressing llms: The truth is rarely pure and never simple.☆23Updated last year
- An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆34Updated 9 months ago
- [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"☆77Updated 9 months ago
- An algorithm for weight-activation quantization (W4A4, W4A8) of LLMs, supporting both static and dynamic quantization☆124Updated 2 months ago
- FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation☆47Updated 8 months ago
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆44Updated 8 months ago
- [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models☆86Updated 10 months ago
- SparseGPT + GPTQ Compression of LLMs like LLaMa, OPT, Pythia☆41Updated 2 years ago
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆121Updated 2 months ago
- EE-LLM is a framework for large-scale training and inference of early-exit (EE) large language models (LLMs).☆56Updated 9 months ago
- ☆122Updated last month
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆141Updated 6 months ago
- Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆86Updated 3 weeks ago
- Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)☆39Updated last year
- The official code for Dropping Backward Propagation (DropBP)☆29Updated 5 months ago
- [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…☆63Updated 11 months ago