minhtannguyen / transformer-mgk
This is the public github for our paper "Transformer with a Mixture of Gaussian Keys"
☆27Updated 2 years ago
Alternatives and similar repositories for transformer-mgk
Users that are interested in transformer-mgk are comparing it to the libraries listed below
Sorting:
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆60Updated 3 years ago
- The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …☆16Updated last year
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆24Updated 2 years ago
- Diffusion based transformer, in PyTorch (Experimental).☆24Updated 2 years ago
- Curse-of-memory phenomenon of RNNs in sequence modelling☆19Updated last week
- ☆21Updated 2 years ago
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆37Updated 3 years ago
- Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012☆50Updated 3 years ago
- Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a two-step attention process wi…☆51Updated 3 years ago
- Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch☆52Updated 4 years ago
- DiWA: Diverse Weight Averaging for Out-of-Distribution Generalization☆30Updated 2 years ago
- Pytorch implementation for "The Surprising Positive Knowledge Transfer in Continual 3D Object Shape Reconstruction"☆33Updated 2 years ago
- ☆23Updated last year
- Code for the PAPA paper☆27Updated 2 years ago
- [NeurIPS 2022] Your Transformer May Not be as Powerful as You Expect (official implementation)☆34Updated last year
- STABILIZING GRADIENTS FOR DEEP NEURAL NETWORKS VIA EFFICIENT SVD PARAMETERIZATION☆16Updated 6 years ago
- Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch☆100Updated 2 years ago
- ☆36Updated 4 years ago
- This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …☆32Updated last year
- Implementation of Multistream Transformers in Pytorch☆54Updated 3 years ago
- Official PyTorch implementation for NeurIPS'24 paper "Knowledge Composition using Task Vectors with Learned Anisotropic Scaling"☆19Updated 2 months ago
- Structured Pruning Adapters in PyTorch☆17Updated last year
- codebase for the SIMAT dataset and evaluation☆39Updated 3 years ago
- Self-Distillation with weighted ground-truth targets; ResNet and Kernel Ridge Regression☆18Updated 3 years ago
- ☆33Updated 4 years ago
- MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space☆41Updated 4 years ago
- PyTorch implementation of CorInfoMax☆21Updated 2 years ago
- Improving Transformation Invariance in Contrastive Representation Learning☆13Updated 4 years ago
- A project to add scalable state-of-the-art out-of-distribution detection (open set recognition) support by changing two lines of code! Pe…☆75Updated 2 years ago
- Official PyTorch implementation of "Energy-Based Contrastive Learning of Visual Representations", NeurIPS 2022 Oral Paper☆10Updated 2 years ago