sooftware / attentionsLinks
PyTorch implementation of some attentions for Deep Learning Researchers.
☆544Updated 3 years ago
Alternatives and similar repositories for attentions
Users that are interested in attentions are comparing it to the libraries listed below
Sorting:
- An implementation of Performer, a linear attention-based transformer, in Pytorch☆1,148Updated 3 years ago
- Transformer based on a variant of attention that is linear complexity in respect to sequence length☆797Updated last year
- Pytorch library for fast transformer implementations☆1,730Updated 2 years ago
- My take on a practical implementation of Linformer for Pytorch.☆419Updated 3 years ago
- Pytorch Lightning code guideline for conferences☆1,279Updated 2 years ago
- An implementation of local windowed attention for language modeling☆475Updated 2 months ago
- Transformer implementation in PyTorch.☆491Updated 6 years ago
- An All-MLP solution for Vision, from Google AI☆1,039Updated 2 months ago
- Learning Rate Warmup in PyTorch☆412Updated 3 months ago
- Reformer, the efficient Transformer, in Pytorch☆2,182Updated 2 years ago
- Unofficial implementation of Google's FNet: Mixing Tokens with Fourier Transforms☆259Updated 4 years ago
- Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch☆430Updated 4 years ago
- Implementation of Linformer for Pytorch☆298Updated last year
- ☆465Updated 2 years ago
- An implementation of 1D, 2D, and 3D positional encoding in Pytorch and TensorFlow☆604Updated 10 months ago
- Early stopping for PyTorch☆1,267Updated 10 months ago
- Long Range Arena for Benchmarking Efficient Transformers☆763Updated last year
- PyTorch implementation of the InfoNCE loss for self-supervised learning.☆588Updated last year
- Flexible components pairing 🤗 Transformers with Pytorch Lightning☆612Updated 2 years ago
- Pytorch reimplementation for "Gradient Surgery for Multi-Task Learning"☆365Updated 4 years ago
- Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch☆749Updated last month
- Contrastive Predictive Coding for Automatic Speaker Verification☆498Updated 5 years ago
- Longformer: The Long-Document Transformer☆2,164Updated 2 years ago
- A learning rate range test implementation in PyTorch☆983Updated 2 months ago
- An (unofficial) implementation of Focal Loss, as described in the RetinaNet paper, generalized to the multi-class case.☆238Updated last year
- Implement the paper "Self-Attention with Relative Position Representations"☆138Updated 4 years ago
- A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch☆228Updated 2 years ago
- An implementation of masked language modeling for Pytorch, made as concise and simple as possible☆179Updated 2 years ago
- Transformers for Longer Sequences☆617Updated 3 years ago
- Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch☆1,171Updated 2 years ago