skolai / fewbit
Compression schema for gradients of activations in backward pass
☆44Updated last year
Alternatives and similar repositories for fewbit:
Users that are interested in fewbit are comparing it to the libraries listed below
- FLOPs and other statistics COunter for Pytorch neural networks☆23Updated 3 years ago
- Learning to Initialize Neural Networks for Stable and Efficient Training☆139Updated 2 years ago
- ☆20Updated 8 months ago
- Official implementation of the paper "You Do Not Fully Utilize Transformer's Representation Capacity"☆28Updated 2 months ago
- Code for the paper "PALBERT: Teaching ALBERT to Ponder", NeurIPS 2022 Spotlight☆37Updated 2 years ago
- ☆71Updated 7 months ago
- ☆18Updated 2 weeks ago
- MUSCO: MUlti-Stage COmpression of neural networks☆71Updated 4 years ago
- PyTorch implementation of L2L execution algorithm☆107Updated 2 years ago
- A neural network training framework within a task-based parallel programming paradigm☆53Updated this week
- ☆60Updated 4 years ago
- RuCLIP tiny (Russian Contrastive Language–Image Pretraining) is a neural network trained to work with different pairs (images, texts).☆33Updated 2 years ago
- Single-line inference of SOTA deep learning models☆29Updated 2 years ago
- ☆15Updated 2 years ago
- Code for MSID, a Multi-Scale Intrinsic Distance for comparing generative models, studying neural networks, and more!☆51Updated 5 years ago
- ☆100Updated 10 months ago
- Customized matrix multiplication kernels☆54Updated 3 years ago
- Lightweight knowledge distillation pipeline☆28Updated 3 years ago
- ☆10Updated 3 years ago
- Russian dialog datasets parsers and crawlers.☆16Updated 3 years ago
- Minimal pretraining script for language modeling in PyTorch. Supporting torch compilation and DDP. It includes a model implementation and…☆12Updated last week
- DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training (ICLR 2023)☆30Updated 2 years ago
- A library for unit scaling in PyTorch☆125Updated 4 months ago
- Hacks for PyTorch☆19Updated 2 years ago
- ☆202Updated 2 years ago
- Deep Generative Models course, 2021☆22Updated 3 years ago
- Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.☆45Updated 9 months ago
- AdaShift optimizer implementation in PyTorch☆17Updated 6 years ago
- ☆14Updated last week
- Latest Weight Averaging (NeurIPS HITY 2022)☆30Updated last year