xl402 / performer
Tensorflow implementation of a linear attention architecture
☆44Updated 3 years ago
Related projects ⓘ
Alternatives and complementary repositories for performer
- Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."☆134Updated 3 years ago
- Simply Numpy implementation of the FAVOR+ attention mechanism, https://teddykoker.com/2020/11/performers/☆37Updated 3 years ago
- Implements MLP-Mixer (https://arxiv.org/abs/2105.01601) with the CIFAR-10 dataset.☆54Updated 2 years ago
- Unofficial PyTorch implementation of Google's FNet: Mixing Tokens with Fourier Transforms. With checkpoints.☆67Updated 2 years ago
- Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning☆155Updated 9 months ago
- Layerwise Batch Entropy Regularization☆22Updated 2 years ago
- Implementation of Feedback Transformer in Pytorch☆104Updated 3 years ago
- Implementation of Fast Transformer in Pytorch☆171Updated 3 years ago
- ☆164Updated last year
- Code for scaling Transformers☆26Updated 3 years ago
- Relative Positional Encoding for Transformers with Linear Complexity☆61Updated 2 years ago
- MTAdam: Automatic Balancing of Multiple Training Loss Terms☆36Updated 3 years ago
- Implementation of modern data augmentation techniques in TensorFlow 2.x to be used in your training pipeline.☆35Updated 4 years ago
- Tensorflow Implementation of "FNet: Mixing Tokens with Fourier Transforms."☆22Updated 3 years ago
- Implementation of Nyström Self-attention, from the paper Nyströmformer☆122Updated 9 months ago
- custom cuda kernel for {2, 3}d relative attention with pytorch wrapper☆43Updated 4 years ago
- Implementation of ETSformer, state of the art time-series Transformer, in Pytorch☆148Updated last year
- This repository contains example code to build models on TPUs☆30Updated last year
- Sequence Modeling with Structured State Spaces☆60Updated 2 years ago
- Axial Positional Embedding for Pytorch☆61Updated 3 years ago
- Implementation of the GBST block from the Charformer paper, in Pytorch☆117Updated 3 years ago
- Code for the Shortformer model, from the ACL 2021 paper by Ofir Press, Noah A. Smith and Mike Lewis.☆145Updated 3 years ago
- Implementation of self-supervised image-level contrastive pretraining methods using Keras.☆69Updated 3 years ago
- a lightweight transformer library for PyTorch☆72Updated 3 years ago
- A 🤗-style implementation of BERT using lambda layers instead of self-attention☆70Updated 4 years ago
- Code for the paper PermuteFormer☆42Updated 3 years ago
- A case study of efficient training of large language models using commodity hardware.☆68Updated 2 years ago
- Adaptive Gradient Clipping☆117Updated 2 years ago
- Implementation of https://arxiv.org/abs/1904.00962☆15Updated 5 years ago
- A collection of Models, Datasets, DataModules, Callbacks, Metrics, Losses and Loggers to better integrate pytorch-lightning with transfor…☆47Updated last year