skolai / fewbitLinks
Compression schema for gradients of activations in backward pass
☆44Updated last year
Alternatives and similar repositories for fewbit
Users that are interested in fewbit are comparing it to the libraries listed below
Sorting:
- ☆20Updated 11 months ago
- ☆60Updated 4 years ago
- AdaShift optimizer implementation in PyTorch☆17Updated 6 years ago
- Learning to Initialize Neural Networks for Stable and Efficient Training☆139Updated 3 years ago
- ☆70Updated 9 months ago
- Official implementation of the paper "You Do Not Fully Utilize Transformer's Representation Capacity"☆28Updated 3 weeks ago
- FLOPs and other statistics COunter for Pytorch neural networks☆23Updated 4 years ago
- MUSCO: MUlti-Stage COmpression of neural networks☆72Updated 4 years ago
- This is the repo for DenseAttention and DANet - fast and conceptually simple modification of standard attention and Transformer☆11Updated last week
- PyTorch implementation of L2L execution algorithm☆107Updated 2 years ago
- ☆15Updated 3 weeks ago
- Code for MSID, a Multi-Scale Intrinsic Distance for comparing generative models, studying neural networks, and more!☆51Updated 6 years ago
- Lightweight knowledge distillation pipeline☆28Updated 3 years ago
- ☆109Updated last year
- Deep Generative Models course, 2021☆22Updated 3 years ago
- NLA 2018 Skoltech course☆55Updated 6 years ago
- Code for the paper "PALBERT: Teaching ALBERT to Ponder", NeurIPS 2022 Spotlight☆37Updated 2 years ago
- ☆18Updated 2 months ago
- Very simple and short implementation of gradient boosting in 18 lines of code☆9Updated 4 years ago
- A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.☆68Updated 10 months ago
- PyTorch implementation of HashedNets☆36Updated 2 years ago
- Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.☆45Updated 11 months ago
- Easy-to-use AdaHessian optimizer (PyTorch)☆79Updated 4 years ago
- ☆15Updated 2 years ago
- sigma-MoE layer☆19Updated last year
- ☆21Updated last month
- FusionBrain Challenge 2.0: creating multimodal multitask model☆16Updated 2 years ago
- Latest Weight Averaging (NeurIPS HITY 2022)☆30Updated 2 years ago
- Faster Pytorch bitsandbytes 4bit fp4 nn.Linear ops☆30Updated last year
- Experiment of using Tangent to autodiff triton☆79Updated last year