sooftware / attentionsLinks
PyTorch implementation of some attentions for Deep Learning Researchers.
☆532Updated 3 years ago
Alternatives and similar repositories for attentions
Users that are interested in attentions are comparing it to the libraries listed below
Sorting:
- Implementation of Linformer for Pytorch☆286Updated last year
- An implementation of Performer, a linear attention-based transformer, in Pytorch☆1,133Updated 3 years ago
- My take on a practical implementation of Linformer for Pytorch.☆414Updated 2 years ago
- Transformer based on a variant of attention that is linear complexity in respect to sequence length☆768Updated last year
- An implementation of local windowed attention for language modeling☆450Updated 4 months ago
- Reformer, the efficient Transformer, in Pytorch☆2,169Updated last year
- Learning Rate Warmup in PyTorch☆410Updated 2 months ago
- An All-MLP solution for Vision, from Google AI☆1,024Updated 8 months ago
- Unofficial implementation of Google's FNet: Mixing Tokens with Fourier Transforms☆260Updated 4 years ago
- Fully featured implementation of Routing Transformer☆292Updated 3 years ago
- Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch☆426Updated 3 years ago
- ☆459Updated 2 years ago
- Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch☆685Updated 6 months ago
- ☆76Updated 5 years ago
- A PyTorch implementation of the 1d and 2d Sinusoidal positional encoding/embedding.☆251Updated 4 years ago
- A Pytorch Implementation of "Attention is All You Need" and "Weighted Transformer Network for Machine Translation"☆557Updated 4 years ago
- Pytorch reimplementation for "Gradient Surgery for Multi-Task Learning"☆344Updated 3 years ago
- Pytorch library for fast transformer implementations☆1,709Updated 2 years ago
- The entmax mapping and its loss, a family of sparse softmax alternatives.☆438Updated 11 months ago
- Transformer implementation in PyTorch.☆487Updated 6 years ago
- Contrastive Predictive Coding for Automatic Speaker Verification☆492Updated 5 years ago
- PyTorch implementation of some learning rate schedulers for deep learning researcher.☆90Updated 2 years ago
- A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch☆227Updated last year
- Implementation of Transformer encoder in PyTorch☆66Updated 4 years ago
- PyTorch implementation of "data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language" from Meta AI☆177Updated 2 years ago
- PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538☆1,111Updated last year
- An implementation of masked language modeling for Pytorch, made as concise and simple as possible☆181Updated last year
- Official PyTorch Implementation of Long-Short Transformer (NeurIPS 2021).☆225Updated 3 years ago
- Experimenting with different regression losses. Implemented in Pytorch.☆147Updated 6 years ago
- pytorch; mask language model ; bert☆72Updated 5 years ago