megvii-research / IntLLaMALinks
IntLLaMA: A fast and light quantization solution for LLaMA
☆18Updated last year
Alternatives and similar repositories for IntLLaMA
Users that are interested in IntLLaMA are comparing it to the libraries listed below
Sorting:
- An object detection codebase based on MegEngine.☆28Updated 2 years ago
- ☆31Updated last year
- 🤗CacheDiT: A Training-free and Easy-to-use Cache Acceleration Toolbox for Diffusion Transformers🔥☆61Updated this week
- Quantized Attention on GPU☆44Updated 7 months ago
- Code for Draft Attention☆72Updated last month
- TVMScript kernel for deformable attention☆25Updated 3 years ago
- ☆11Updated last year
- FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation [Efficient ML Model]☆24Updated 3 weeks ago
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆38Updated last year
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆25Updated last year
- High Performance FP8 GEMM Kernels for SM89 and later GPUs.☆14Updated 5 months ago
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆41Updated last month
- Odysseus: Playground of LLM Sequence Parallelism☆70Updated last year
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆27Updated last year
- An auxiliary project analysis of the characteristics of KV in DiT Attention.☆31Updated 6 months ago
- Patch convolution to avoid large GPU memory usage of Conv2D☆88Updated 5 months ago
- Various test models in WNNX format. It can view with `pip install wnetron && wnetron`☆12Updated 3 years ago
- [CVPR 2025] Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers☆51Updated 9 months ago
- ☆21Updated 2 months ago
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- ☆21Updated 2 years ago
- Benchmarking Attention Mechanism in Vision Transformers.☆18Updated 2 years ago
- [ECCV24] MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization☆42Updated 7 months ago
- # Unified Normalization (ACM MM'22) By Qiming Yang, Kai Zhang, Chaoxiang Lan, Zhi Yang, Zheyang Li, Wenming Tan, Jun Xiao, and Shiliang P…☆34Updated 2 years ago
- torch_quantizer is a out-of-box quantization tool for PyTorch models on CUDA backend, specially optimized for Diffusion Models.☆22Updated last year
- Training LLaMA language model with MMEngine! It supports LoRA fine-tuning!☆40Updated 2 years ago
- ☆32Updated 2 weeks ago
- [TMLR] Official PyTorch implementation of paper "Efficient Quantization-aware Training with Adaptive Coreset Selection"☆33Updated 10 months ago
- Self Reproduction Code of Paper "Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (MIT CSAIL)☆16Updated last year
- Official PyTorch implementation of "Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models"☆36Updated 3 weeks ago