aredden / torch-bnb-fp4Links
Faster Pytorch bitsandbytes 4bit fp4 nn.Linear ops
☆30Updated last year
Alternatives and similar repositories for torch-bnb-fp4
Users that are interested in torch-bnb-fp4 are comparing it to the libraries listed below
Sorting:
- ☆118Updated last year
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- ☆159Updated last year
- This repository contains the experimental PyTorch native float8 training UX☆224Updated last year
- Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.☆46Updated last year
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆174Updated 2 months ago
- Official PyTorch implementation of "GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance" (ICML 2025)☆43Updated last month
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆73Updated last year
- QuIP quantization☆57Updated last year
- ☆140Updated 6 months ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆80Updated 11 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆76Updated last year
- A library for unit scaling in PyTorch☆129Updated last month
- ☆29Updated last year
- Experiment of using Tangent to autodiff triton☆80Updated last year
- This repository contains code for the MicroAdam paper.☆19Updated 8 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆142Updated last year
- Low-bit optimizers for PyTorch☆130Updated last year
- Low-Rank Llama Custom Training☆23Updated last year
- ☆154Updated 2 years ago
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆49Updated 2 years ago
- The evaluation framework for training-free sparse attention in LLMs☆91Updated 2 months ago
- ☆149Updated 2 months ago
- ☆81Updated 7 months ago
- Reorder-based post-training quantization for large language model☆195Updated 2 years ago
- ☆32Updated last year
- ☆149Updated 2 years ago
- ☆110Updated last year
- A block oriented training approach for inference time optimization.☆34Updated last year
- A bunch of kernels that might make stuff slower 😉☆58Updated last week