skolai / fewbitLinks
Compression schema for gradients of activations in backward pass
☆44Updated 2 years ago
Alternatives and similar repositories for fewbit
Users that are interested in fewbit are comparing it to the libraries listed below
Sorting:
- Learning to Initialize Neural Networks for Stable and Efficient Training☆139Updated 3 years ago
- PyTorch implementation of L2L execution algorithm☆108Updated 2 years ago
- Customized matrix multiplication kernels☆57Updated 3 years ago
- ☆60Updated 5 years ago
- ☆121Updated last year
- A library for unit scaling in PyTorch☆132Updated 3 months ago
- PyTorch implementation of HashedNets☆37Updated 2 years ago
- ☆36Updated last year
- ☆219Updated 2 years ago
- Latest Weight Averaging (NeurIPS HITY 2022)☆31Updated 2 years ago
- [JMLR'20] NeurIPS 2019 MicroNet Challenge Efficient Language Modeling, Champion☆41Updated 4 years ago
- Easy-to-use AdaHessian optimizer (PyTorch)☆79Updated 4 years ago
- Hacks for PyTorch☆19Updated 2 years ago
- Official code for "Distributed Deep Learning in Open Collaborations" (NeurIPS 2021)☆117Updated 3 years ago
- ☆70Updated last year
- Implementation of fused cosine similarity attention in the same style as Flash Attention☆217Updated 2 years ago
- sigma-MoE layer☆20Updated last year
- Dynamic Neural Architecture Search Toolkit☆30Updated 10 months ago
- Experiment of using Tangent to autodiff triton☆80Updated last year
- ☆45Updated last week
- The official implementation of the ChordMixer architecture.☆61Updated 2 years ago
- Code for the paper "Secure Distributed Training at Scale" (ICML 2022)☆16Updated 8 months ago
- ☆18Updated 6 months ago
- ☆29Updated 3 years ago
- Simple and efficient RevNet-Library for PyTorch with XLA and DeepSpeed support and parameter offload☆131Updated 3 years ago
- Faster Pytorch bitsandbytes 4bit fp4 nn.Linear ops☆29Updated last year
- MUSCO: MUlti-Stage COmpression of neural networks☆72Updated 4 years ago
- Memory-efficient transformer. Work in progress.☆19Updated 3 years ago
- ☆75Updated 2 years ago
- Block-sparse primitives for PyTorch☆160Updated 4 years ago