Vahe1994 / AQLM
Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852
☆1,170Updated last week
Related projects ⓘ
Alternatives and complementary repositories for AQLM
- ☆892Updated last month
- Training LLMs with QLoRA + FSDP☆1,418Updated last week
- Reaching LLaMA2 Performance with 0.1M Dollars☆960Updated 3 months ago
- Accelerate your Hugging Face Transformers 7.6-9x. Native to Hugging Face and PyTorch.☆687Updated 2 months ago
- GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection☆1,435Updated 3 weeks ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆701Updated last week
- The official implementation of Self-Play Fine-Tuning (SPIN)☆1,045Updated 6 months ago
- A simple, performant and scalable Jax LLM!☆1,532Updated this week
- [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding☆1,149Updated last month
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆1,765Updated this week
- Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch☆1,692Updated last week
- ☆505Updated 3 weeks ago
- Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI☆1,336Updated 7 months ago
- A native PyTorch Library for large model training☆2,623Updated this week
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆685Updated this week
- PyTorch native quantization and sparsity for training and inference☆1,585Updated this week
- DeepSeek-VL: Towards Real-World Vision-Language Understanding☆2,077Updated 6 months ago
- ☆448Updated 7 months ago
- For releasing code related to compression methods for transformers, accompanying our publications☆372Updated last month
- Large-scale LLM inference engine☆1,134Updated this week
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆523Updated this week
- Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)☆826Updated this week
- Minimalistic large language model 3D-parallelism training☆1,260Updated this week
- [ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling☆1,529Updated 4 months ago
- Train Models Contrastively in Pytorch☆546Updated this week
- Make PyTorch models up to 40% faster! Thunder is a source to source compiler for PyTorch. It enables using different hardware executors a…☆1,199Updated this week
- A throughput-oriented high-performance serving framework for LLMs☆636Updated 2 months ago
- A pytorch quantization backend for optimum☆824Updated last week
- [NeurIPS'24 Spotlight] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces in…☆791Updated this week
- Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"☆803Updated 3 months ago