kssteven418 / SqueezeLLM-gradients
☆14Updated 9 months ago
Related projects ⓘ
Alternatives and complementary repositories for SqueezeLLM-gradients
- [NeurIPS'23] Speculative Decoding with Big Little Decoder☆86Updated 9 months ago
- The official PyTorch implementation of the NeurIPS2022 (spotlight) paper, Outlier Suppression: Pushing the Limit of Low-bit Transformer L…☆46Updated 2 years ago
- Official Pytorch Implementation of "Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity"☆51Updated 4 months ago
- An algorithm for static activation quantization of LLMs☆79Updated 2 weeks ago
- AFPQ code implementation☆18Updated last year
- GPU operators for sparse tensor operations☆29Updated 8 months ago
- ☆24Updated 7 months ago
- ☆41Updated 2 years ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆149Updated 4 months ago
- ☆20Updated last week
- QAQ: Quality Adaptive Quantization for LLM KV Cache☆42Updated 7 months ago
- Code Repository of Evaluating Quantized Large Language Models☆103Updated 2 months ago
- Code for the AAAI 2024 Oral paper "OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Model…☆53Updated 8 months ago
- Official PyTorch implementation of FlatQuant: Flatness Matters for LLM Quantization☆66Updated last week
- 16-fold memory access reduction with nearly no loss☆58Updated 2 weeks ago
- Efficient LLM Inference Acceleration using Prompting☆44Updated last month
- Activation-aware Singular Value Decomposition for Compressing Large Language Models☆49Updated last month
- ☆90Updated 2 months ago
- ☆34Updated 9 months ago
- [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.☆85Updated 6 months ago
- [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache☆243Updated last month
- ☆47Updated last year
- Official implementation of the EMNLP23 paper: Outlier Suppression+: Accurate quantization of large language models by equivalent and opti…☆42Updated last year
- ☆47Updated 2 months ago
- Code repo for the paper "SpinQuant LLM quantization with learned rotations"☆164Updated last week
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆38Updated 10 months ago
- Sirius, an efficient correction mechanism, which significantly boosts Contextual Sparsity models on reasoning tasks while maintaining its…☆19Updated 2 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆203Updated this week
- ☆132Updated 4 months ago
- Code for the NeurIPS 2022 paper "Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning".☆104Updated last year