sooftware / attentions
PyTorch implementation of some attentions for Deep Learning Researchers.
☆523Updated 2 years ago
Alternatives and similar repositories for attentions:
Users that are interested in attentions are comparing it to the libraries listed below
- An implementation of Performer, a linear attention-based transformer, in Pytorch☆1,114Updated 3 years ago
- Pytorch library for fast transformer implementations☆1,679Updated last year
- Transformer based on a variant of attention that is linear complexity in respect to sequence length☆738Updated 9 months ago
- Implementation of Linformer for Pytorch☆266Updated last year
- Reformer, the efficient Transformer, in Pytorch☆2,152Updated last year
- Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch☆639Updated 2 months ago
- Learning Rate Warmup in PyTorch☆403Updated 2 weeks ago
- An All-MLP solution for Vision, from Google AI☆1,014Updated 5 months ago
- An implementation of local windowed attention for language modeling☆420Updated last month
- Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch☆426Updated 3 years ago
- Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch☆1,120Updated last year
- Pytorch Lightning code guideline for conferences☆1,253Updated last year
- Contrastive Predictive Coding for Automatic Speaker Verification☆486Updated 5 years ago
- My take on a practical implementation of Linformer for Pytorch.☆412Updated 2 years ago
- ☆450Updated last year
- An implementation of 1D, 2D, and 3D positional encoding in Pytorch and TensorFlow☆566Updated 3 months ago
- A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch☆221Updated last year
- An implementation of masked language modeling for Pytorch, made as concise and simple as possible☆178Updated last year
- Unofficial implementation of Google's FNet: Mixing Tokens with Fourier Transforms☆258Updated 3 years ago
- Transformer implementation in PyTorch.☆479Updated 5 years ago
- PyTorch implementation of some learning rate schedulers for deep learning researcher.☆89Updated 2 years ago
- Standalone TFRecord reader/writer with PyTorch data loaders☆874Updated 6 months ago
- Flexible components pairing 🤗 Transformers with Pytorch Lightning☆612Updated 2 years ago
- An (unofficial) implementation of Focal Loss, as described in the RetinaNet paper, generalized to the multi-class case.☆230Updated last year
- PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538☆1,043Updated 10 months ago
- PyTorch implementation of the InfoNCE loss for self-supervised learning.☆520Updated last year
- Pytorch reimplementation for "Gradient Surgery for Multi-Task Learning"☆318Updated 3 years ago
- Long Range Arena for Benchmarking Efficient Transformers☆745Updated last year
- Fully featured implementation of Routing Transformer☆288Updated 3 years ago
- Source code for "On the Relationship between Self-Attention and Convolutional Layers"☆1,096Updated 2 years ago