aiha-lab / TernGEMM
TernGEMM: General Matrix Multiply Library with Ternary Weights for Fast DNN Inference
☆13Updated 2 years ago
Related projects: ⓘ
- Layer-wise Pruning of Transformer Heads for Efficient Language Modeling☆19Updated 2 years ago
- [ICASSP'22] Integer-only Zero-shot Quantization for Efficient Speech Recognition☆30Updated 2 years ago
- ☆33Updated 5 months ago
- ☆14Updated 7 months ago
- The official PyTorch implementation of the NeurIPS2022 (spotlight) paper, Outlier Suppression: Pushing the Limit of Low-bit Transformer L…☆46Updated last year
- In this repository, we explore model compression for transformer architectures via quantization. We specifically explore quantization awa…☆22Updated 3 years ago
- [NeurIPS'23] Speculative Decoding with Big Little Decoder☆84Updated 7 months ago
- Official Repo for EdgeQAT☆12Updated 6 months ago
- ☆127Updated last month
- PyTorch extension for emulating FP8 data formats on standard FP32 Xeon/GPU hardware.☆98Updated 9 months ago
- Code repo for the paper "SpinQuant LLM quantization with learned rotations"☆79Updated this week
- ☆186Updated 2 years ago
- AFPQ code implementation☆15Updated 10 months ago
- [ICML'21 Oral] I-BERT: Integer-only BERT Quantization☆220Updated last year
- Fast Hadamard transform in CUDA, with a PyTorch interface☆87Updated 3 months ago
- Code Repository of Evaluating Quantized Large Language Models☆89Updated last week
- ☆15Updated this week
- This repository contains the results and code for the MLPerf™ Tiny Inference v0.7 benchmark.☆14Updated last year
- Code for the AAAI 2024 Oral paper "OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Model…☆50Updated 6 months ago
- ☆15Updated last year
- High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.☆81Updated 2 months ago
- [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.☆69Updated 4 months ago
- ☆151Updated last year
- Forward and backward Attention DNN operators implementationed by LibTorch, cuDNN, and Eigen.☆27Updated last year
- ☆113Updated last year
- ☆102Updated 3 months ago
- ☆18Updated 2 years ago
- An innovative method expediting LLMs via streamlined semi-autoregressive generation and draft verification.☆21Updated 7 months ago
- QQQ is an innovative and hardware-optimized W4A8 quantization solution.☆59Updated last week
- [NeurIPS 2023] Token-Scaled Logit Distillation for Ternary Weight Generative Language Models☆15Updated 9 months ago