sooftware / attentions
PyTorch implementation of some attentions for Deep Learning Researchers.
☆532Updated 3 years ago
Alternatives and similar repositories for attentions
Users that are interested in attentions are comparing it to the libraries listed below
Sorting:
- An implementation of Performer, a linear attention-based transformer, in Pytorch☆1,129Updated 3 years ago
- Transformer based on a variant of attention that is linear complexity in respect to sequence length☆764Updated last year
- Reformer, the efficient Transformer, in Pytorch☆2,168Updated last year
- Pytorch library for fast transformer implementations☆1,703Updated 2 years ago
- My take on a practical implementation of Linformer for Pytorch.☆415Updated 2 years ago
- Implementation of Linformer for Pytorch☆286Updated last year
- Learning Rate Warmup in PyTorch☆410Updated 2 months ago
- An implementation of local windowed attention for language modeling☆449Updated 4 months ago
- Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch☆673Updated 5 months ago
- Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch☆1,142Updated last year
- Early stopping for PyTorch☆1,253Updated 6 months ago
- Contrastive Predictive Coding for Automatic Speaker Verification☆492Updated 5 years ago
- Pytorch reimplementation for "Gradient Surgery for Multi-Task Learning"☆339Updated 3 years ago
- An implementation of masked language modeling for Pytorch, made as concise and simple as possible☆180Updated last year
- PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538☆1,102Updated last year
- Pytorch Implementation of Google BERT☆593Updated 5 years ago
- An All-MLP solution for Vision, from Google AI☆1,022Updated 8 months ago
- Long Range Arena for Benchmarking Efficient Transformers☆754Updated last year
- Transformers for Longer Sequences☆608Updated 2 years ago
- ☆459Updated 2 years ago
- Pytorch Lightning code guideline for conferences☆1,264Updated last year
- Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch☆425Updated 3 years ago
- A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch☆226Updated last year
- Structured state space sequence models☆2,621Updated 9 months ago
- Flexible components pairing 🤗 Transformers with Pytorch Lightning☆608Updated 2 years ago
- Implement the paper "Self-Attention with Relative Position Representations"☆128Updated 4 years ago
- An implementation of 1D, 2D, and 3D positional encoding in Pytorch and TensorFlow☆584Updated 6 months ago
- PyTorch implementation of "data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language" from Meta AI☆177Updated last year
- Longformer: The Long-Document Transformer☆2,120Updated 2 years ago
- A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models☆742Updated last year