Separius / awesome-fast-attentionLinks
list of efficient attention modules
☆1,011Updated 3 years ago
Alternatives and similar repositories for awesome-fast-attention
Users that are interested in awesome-fast-attention are comparing it to the libraries listed below
Sorting:
- [ICLR 2020] Lite Transformer with Long-Short Range Attention☆610Updated 11 months ago
- Some tricks of pytorch...☆1,194Updated last year
- Debug PyTorch code using PySnooper☆798Updated 4 years ago
- Gradually-Warmup Learning Rate Scheduler for PyTorch☆989Updated 8 months ago
- some tircks for PyTorch☆576Updated 5 years ago
- PyTorch implementation of Contrastive Learning methods☆1,985Updated last year
- My best practice of training large dataset using PyTorch.☆1,100Updated last year
- DeLighT: Very Deep and Light-Weight Transformers☆469Updated 4 years ago
- My take on a practical implementation of Linformer for Pytorch.☆414Updated 2 years ago
- [arXiv 2019] "Contrastive Multiview Coding", also contains implementations for MoCo and InstDis☆1,325Updated 4 years ago
- Sublinear memory optimization for deep learning. https://arxiv.org/abs/1604.06174☆598Updated 5 years ago
- Reformer, the efficient Transformer, in Pytorch☆2,171Updated 2 years ago
- Source code for "On the Relationship between Self-Attention and Convolutional Layers"☆1,103Updated 2 years ago
- This repository contains code for the paper "Decoupling Representation and Classifier for Long-Tailed Recognition", published at ICLR 202…☆972Updated 3 years ago
- ☆879Updated last year
- A quickstart and benchmark for pytorch distributed training.☆1,668Updated 11 months ago
- label-smooth, amsoftmax, partial-fc, focal-loss, triplet-loss, lovasz-softmax. Maybe useful☆2,243Updated 8 months ago
- [CVPR 2021] Involution: Inverting the Inherence of Convolution for Visual Recognition, a brand new neural operator☆1,312Updated 3 years ago
- An All-MLP solution for Vision, from Google AI☆1,025Updated 9 months ago
- Pytorch library for fast transformer implementations☆1,718Updated 2 years ago
- Ranger - a synergistic optimizer using RAdam (Rectified Adam), Gradient Centralization and LookAhead in one codebase☆1,202Updated last year
- Pytorch implementation of the paper "Class-Balanced Loss Based on Effective Number of Samples"☆798Updated last year
- A comprehensive list of awesome contrastive self-supervised learning papers.☆1,277Updated 9 months ago
- Implementing Attention Augmented Convolutional Networks using Pytorch☆654Updated 3 years ago
- pytorch memory track code☆1,019Updated 4 years ago
- Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。☆2,602Updated 2 years ago
- lookahead optimizer (Lookahead Optimizer: k steps forward, 1 step back) for pytorch☆337Updated 5 years ago
- An implementation of Performer, a linear attention-based transformer, in Pytorch☆1,134Updated 3 years ago
- Model analyzer in PyTorch☆1,485Updated 2 years ago
- Official PyTorch Repo for "ReZero is All You Need: Fast Convergence at Large Depth"☆408Updated 11 months ago