megvii-research / IntLLaMA
IntLLaMA: A fast and light quantization solution for LLaMA
☆18Updated last year
Related projects ⓘ
Alternatives and complementary repositories for IntLLaMA
- ☆32Updated this week
- An object detection codebase based on MegEngine.☆28Updated last year
- PyTorch code for Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers☆34Updated 2 months ago
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆19Updated 8 months ago
- TVMScript kernel for deformable attention☆24Updated 2 years ago
- ☆29Updated 5 months ago
- Patch convolution to avoid large GPU memory usage of Conv2D☆79Updated 5 months ago
- [EMNLP 2024] RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization☆21Updated last month
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆36Updated 8 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆57Updated 5 months ago
- ☆20Updated 2 years ago
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆20Updated 5 months ago
- ☆46Updated 2 months ago
- The official implementation of the paper "Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation"☆16Updated 4 months ago
- Benchmarking Attention Mechanism in Vision Transformers.☆16Updated 2 years ago
- torch_quantizer is a out-of-box quantization tool for PyTorch models on CUDA backend, specially optimized for Diffusion Models.☆18Updated 7 months ago
- ACL 2023☆38Updated last year
- A light-weight and high-efficient training framework for accelerating diffusion tasks.☆41Updated 2 months ago
- Official Code For Dual Grained Quantization: Efficient Fine-Grained Quantization for LLM☆13Updated 10 months ago
- ☆11Updated last year
- Quantized Attention on GPU☆30Updated 2 weeks ago
- GPTQ inference TVM kernel☆36Updated 6 months ago
- ☆15Updated 3 weeks ago
- ☆35Updated 2 weeks ago
- Page for the CVPR 2023 Tutorial - Efficient Neural Networks: From Algorithm Design to Practical Mobile Deployments☆13Updated last year
- A toolkit for developers to simplify the transformation of nn.Module instances. It's now corresponding to Pytorch.fx.☆13Updated last year
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆43Updated last year
- APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…☆20Updated last week
- BESA is a differentiable weight pruning technique for large language models.☆14Updated 8 months ago