aredden / torch-bnb-fp4Links
Faster Pytorch bitsandbytes 4bit fp4 nn.Linear ops
☆30Updated last year
Alternatives and similar repositories for torch-bnb-fp4
Users that are interested in torch-bnb-fp4 are comparing it to the libraries listed below
Sorting:
- This repository contains the experimental PyTorch native float8 training UX☆224Updated last year
- ☆158Updated last year
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- ☆114Updated last year
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆71Updated last year
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆152Updated last month
- Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.☆46Updated last year
- research impl of Native Sparse Attention (2502.11089)☆60Updated 5 months ago
- Experiment of using Tangent to autodiff triton☆80Updated last year
- Boosting 4-bit inference kernels with 2:4 Sparsity☆80Updated 11 months ago
- A library for unit scaling in PyTorch☆128Updated 3 weeks ago
- QuIP quantization☆55Updated last year
- Low-bit optimizers for PyTorch☆130Updated last year
- Official PyTorch implementation of "GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance" (ICML 2025)☆39Updated last month
- The evaluation framework for training-free sparse attention in LLMs☆88Updated last month
- A block oriented training approach for inference time optimization.☆33Updated 11 months ago
- ☆137Updated 5 months ago
- A place to store reusable transformer components of my own creation or found on the interwebs☆59Updated last week
- A safetensors extension to efficiently store sparse quantized tensors on disk☆142Updated this week
- This repository contains code for the MicroAdam paper.☆19Updated 7 months ago
- Load compute kernels from the Hub☆220Updated last week
- DPO, but faster 🚀☆44Updated 8 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆140Updated 11 months ago
- ☆154Updated 2 years ago
- Odysseus: Playground of LLM Sequence Parallelism☆72Updated last year
- ☆83Updated last year
- ☆145Updated last month
- Repository for CPU Kernel Generation for LLM Inference☆26Updated 2 years ago
- PB-LLM: Partially Binarized Large Language Models☆153Updated last year
- ☆29Updated last year