minhtannguyen / transformer-mgkLinks
This is the public github for our paper "Transformer with a Mixture of Gaussian Keys"
☆28Updated 2 years ago
Alternatives and similar repositories for transformer-mgk
Users that are interested in transformer-mgk are comparing it to the libraries listed below
Sorting:
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆62Updated 3 years ago
- Code for the PAPA paper☆27Updated 2 years ago
- Curse-of-memory phenomenon of RNNs in sequence modelling☆19Updated 2 months ago
- [NeurIPS 2022] Your Transformer May Not be as Powerful as You Expect (official implementation)☆34Updated last year
- ☆37Updated 11 months ago
- The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …☆16Updated last month
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆39Updated 3 years ago
- HGRN2: Gated Linear RNNs with State Expansion☆55Updated 10 months ago
- Recycling diverse models☆45Updated 2 years ago
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆24Updated 2 years ago
- A project to improve out-of-distribution detection (open set recognition) and uncertainty estimation by changing a few lines of code in y…☆45Updated 2 years ago
- Deep Learning & Information Bottleneck☆61Updated 2 years ago
- Code for "Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?" [ICML 2023]☆36Updated 10 months ago
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆75Updated last year
- ☆21Updated 2 years ago
- Code for paper "Can contrastive learning avoid shortcut solutions?" NeurIPS 2021.☆47Updated 3 years ago
- ☆16Updated 2 years ago
- Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012☆49Updated 3 years ago
- Latest Weight Averaging (NeurIPS HITY 2022)☆30Updated 2 years ago
- Diffusion based transformer, in PyTorch (Experimental).☆24Updated 2 years ago
- [ICLR 2021] "Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning" by Tianlong Chen*, Zhenyu Zhang*, Sijia Liu, S…☆25Updated 3 years ago
- ☆12Updated 2 years ago
- Implementation of Memformer, a Memory-augmented Transformer, in Pytorch☆119Updated 4 years ago
- Official Code for ICLR 2024 Paper: Non-negative Contrastive Learning☆45Updated last year
- Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch☆53Updated 4 years ago
- This is the official implementation for Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models.☆28Updated 2 years ago
- Position Prediction as an Effective Pretraining Strategy☆8Updated 2 years ago
- DiWA: Diverse Weight Averaging for Out-of-Distribution Generalization☆31Updated 2 years ago
- ☆109Updated 2 years ago
- Structured Pruning Adapters in PyTorch☆18Updated last year