aredden / torch-bnb-fp4Links
Faster Pytorch bitsandbytes 4bit fp4 nn.Linear ops
☆30Updated last year
Alternatives and similar repositories for torch-bnb-fp4
Users that are interested in torch-bnb-fp4 are comparing it to the libraries listed below
Sorting:
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- ☆109Updated last year
- ☆157Updated last year
- QuIP quantization☆54Updated last year
- Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.☆45Updated 11 months ago
- Repository for CPU Kernel Generation for LLM Inference☆26Updated last year
- Boosting 4-bit inference kernels with 2:4 Sparsity☆79Updated 9 months ago
- This repository contains the experimental PyTorch native float8 training UX☆224Updated 10 months ago
- research impl of Native Sparse Attention (2502.11089)☆54Updated 4 months ago
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆131Updated last week
- Work in progress.☆69Updated 2 weeks ago
- Experiment of using Tangent to autodiff triton☆79Updated last year
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆48Updated 2 years ago
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- ☆130Updated 4 months ago
- ☆75Updated 5 months ago
- Low-Rank Llama Custom Training☆23Updated last year
- A place to store reusable transformer components of my own creation or found on the interwebs☆56Updated last week
- Odysseus: Playground of LLM Sequence Parallelism☆70Updated last year
- 🎬 3.7× faster video generation E2E 🖼️ 1.6× faster image generation E2E ⚡ ColumnSparseAttn 9.3× vs FlashAttn‑3 💨 ColumnSparseGEMM 2.5× …☆74Updated last week
- ☆31Updated last year
- PB-LLM: Partially Binarized Large Language Models☆152Updated last year
- ☆105Updated 10 months ago
- A block oriented training approach for inference time optimization.☆33Updated 10 months ago
- Fast, Modern, and Low Precision PyTorch Optimizers☆94Updated this week
- A safetensors extension to efficiently store sparse quantized tensors on disk☆129Updated this week
- Simple implementation of Speculative Sampling in NumPy for GPT-2.☆95Updated last year
- ☆28Updated 11 months ago
- A library for unit scaling in PyTorch☆125Updated 6 months ago
- Low-bit optimizers for PyTorch☆129Updated last year