megvii-research / IntLLaMA
IntLLaMA: A fast and light quantization solution for LLaMA
☆18Updated last year
Alternatives and similar repositories for IntLLaMA:
Users that are interested in IntLLaMA are comparing it to the libraries listed below
- An object detection codebase based on MegEngine.☆28Updated 2 years ago
- ☆30Updated 11 months ago
- Quantized Attention on GPU☆45Updated 5 months ago
- [CVPR 2025] Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers☆47Updated 8 months ago
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆40Updated this week
- Triton implement of bi-directional (non-causal) linear attention☆46Updated 3 months ago
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆25Updated 10 months ago
- TVMScript kernel for deformable attention☆25Updated 3 years ago
- APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…☆23Updated last week
- An auxiliary project analysis of the characteristics of KV in DiT Attention.☆29Updated 5 months ago
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆36Updated last year
- Official Code For Dual Grained Quantization: Efficient Fine-Grained Quantization for LLM☆14Updated last year
- GPTQ inference TVM kernel☆38Updated last year
- Patch convolution to avoid large GPU memory usage of Conv2D☆86Updated 3 months ago
- Official PyTorch implementation of "Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models"☆30Updated last week
- ☆11Updated last year
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆27Updated last year
- Benchmark tests supporting the TiledCUDA library.☆16Updated 5 months ago
- torch_quantizer is a out-of-box quantization tool for PyTorch models on CUDA backend, specially optimized for Diffusion Models.☆22Updated last year
- [ICML 2022] "DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks", by Yonggan …☆71Updated 2 years ago
- DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling☆15Updated this week
- Training LLaMA language model with MMEngine! It supports LoRA fine-tuning!☆40Updated 2 years ago
- Benchmarking Attention Mechanism in Vision Transformers.☆17Updated 2 years ago
- [CVPR-2023] Towards Any Structural Pruning☆16Updated 2 years ago
- Odysseus: Playground of LLM Sequence Parallelism☆69Updated 10 months ago
- ☆16Updated last year
- [TMLR] Official PyTorch implementation of paper "Efficient Quantization-aware Training with Adaptive Coreset Selection"☆30Updated 8 months ago
- ACL 2023☆39Updated last year
- # Unified Normalization (ACM MM'22) By Qiming Yang, Kai Zhang, Chaoxiang Lan, Zhi Yang, Zheyang Li, Wenming Tan, Jun Xiao, and Shiliang P…☆34Updated 2 years ago
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆48Updated last year