SqueezeBits / QUICKLinks
QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference
☆118Updated last year
Alternatives and similar repositories for QUICK
Users that are interested in QUICK are comparing it to the libraries listed below
Sorting:
- [MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving☆318Updated last year
- ☆79Updated 6 months ago
- An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).☆260Updated 2 weeks ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆165Updated last year
- [NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization☆365Updated 11 months ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆80Updated 10 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆106Updated 2 months ago
- Applied AI experiments and examples for PyTorch☆289Updated 2 months ago
- [ICML 2024 Oral] Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs☆111Updated last month
- Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.☆193Updated this week
- This repository contains integer operators on GPUs for PyTorch.☆208Updated last year
- ☆136Updated 5 months ago
- Fast low-bit matmul kernels in Triton☆338Updated this week
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity☆216Updated last year
- [ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache