megvii-research / IntLLaMALinks
IntLLaMA: A fast and light quantization solution for LLaMA
☆18Updated 2 years ago
Alternatives and similar repositories for IntLLaMA
Users that are interested in IntLLaMA are comparing it to the libraries listed below
Sorting:
- An object detection codebase based on MegEngine.☆28Updated 2 years ago
- TVMScript kernel for deformable attention☆25Updated 3 years ago
- ☆32Updated last year
- Quantized Attention on GPU☆44Updated 10 months ago
- Patch convolution to avoid large GPU memory usage of Conv2D☆92Updated 8 months ago
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆30Updated last year
- ☆22Updated 2 years ago
- Benchmarking Attention Mechanism in Vision Transformers.☆18Updated 2 years ago
- torch.compile artifacts for common deep learning models, can be used as a learning resource for torch.compile☆18Updated last year
- Odysseus: Playground of LLM Sequence Parallelism☆77Updated last year
- Training LLaMA language model with MMEngine! It supports LoRA fine-tuning!☆41Updated 2 years ago
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆38Updated last year
- ☆12Updated 2 years ago
- ☆98Updated 4 months ago
- High Performance FP8 GEMM Kernels for SM89 and later GPUs.☆20Updated 8 months ago
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆25Updated 2 months ago
- torch_quantizer is a out-of-box quantization tool for PyTorch models on CUDA backend, specially optimized for Diffusion Models.☆22Updated last year
- ☆33Updated 3 months ago
- [ECCV 2022] SuperTickets: Drawing Task-Agnostic Lottery Tickets from Supernets via Jointly Architecture Searching and Parameter Pruning☆21Updated 3 years ago
- [TMLR] Official PyTorch implementation of paper "Efficient Quantization-aware Training with Adaptive Coreset Selection"☆34Updated last year
- Persistent dense gemm for Hopper in `CuTeDSL`☆15Updated last month
- ☆24Updated 6 months ago
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆43Updated 3 months ago
- [COLM 2025] Official PyTorch implementation of "Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models"☆54Updated 2 months ago
- [ICML 2022] "DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks", by Yonggan …☆72Updated 3 years ago
- Benchmark tests supporting the TiledCUDA library.☆17Updated 10 months ago
- Triton implement of bi-directional (non-causal) linear attention☆56Updated 8 months ago
- [CVPR 2024] DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model☆17Updated last year
- FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation☆51Updated last month
- APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…☆24Updated 3 months ago