minhtannguyen / transformer-mgkLinks
This is the public github for our paper "Transformer with a Mixture of Gaussian Keys"
☆27Updated 2 years ago
Alternatives and similar repositories for transformer-mgk
Users that are interested in transformer-mgk are comparing it to the libraries listed below
Sorting:
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆61Updated 3 years ago
- Curse-of-memory phenomenon of RNNs in sequence modelling☆19Updated 3 weeks ago
- Code for the PAPA paper☆27Updated 2 years ago
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆39Updated 3 years ago
- The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …☆16Updated last year
- ☆21Updated 2 years ago
- Diffusion based transformer, in PyTorch (Experimental).☆24Updated 2 years ago
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆24Updated 2 years ago
- [NeurIPS 2022] Your Transformer May Not be as Powerful as You Expect (official implementation)☆34Updated last year
- Github code for the paper Maximum Class Separation as Inductive Bias in One Matrix. Arxiv link: https://arxiv.org/abs/2206.08704☆29Updated 2 years ago
- ☆20Updated 2 years ago
- User-friendly implementation of the Mixture-of-Sparse-Attention (MoSA). MoSA selects distinct tokens for each head with expert choice rou…☆17Updated last month
- Distributional Sliced-Wasserstein distance code☆50Updated 10 months ago
- Code for the paper "Data Feedback Loops: Model-driven Amplification of Dataset Biases"☆16Updated 2 years ago
- Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012☆50Updated 3 years ago
- HGRN2: Gated Linear RNNs with State Expansion☆54Updated 9 months ago
- ☆23Updated 8 months ago
- This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …☆32Updated last year
- Recycling diverse models☆44Updated 2 years ago
- Stochastic Multiple Target Sampling Gradient Descent (NeurIPS 2022)☆13Updated 2 years ago
- Implementation of Memformer, a Memory-augmented Transformer, in Pytorch☆117Updated 4 years ago
- ☆13Updated 3 years ago
- A regularized self-labeling approach to improve the generalization and robustness of fine-tuned models☆28Updated 3 years ago
- Weighted Training for Cross-Task Learning☆15Updated 2 years ago
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆73Updated last year
- Latest Weight Averaging (NeurIPS HITY 2022)☆30Updated last year
- DiWA: Diverse Weight Averaging for Out-of-Distribution Generalization☆31Updated 2 years ago
- Implementation of Multistream Transformers in Pytorch☆54Updated 3 years ago
- Meta-Album meta-dataset for few-shot image classification☆24Updated 2 years ago
- ☆26Updated 3 years ago