minhtannguyen / transformer-mgkLinks
This is the public github for our paper "Transformer with a Mixture of Gaussian Keys"
☆28Updated 3 years ago
Alternatives and similar repositories for transformer-mgk
Users that are interested in transformer-mgk are comparing it to the libraries listed below
Sorting:
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆63Updated 3 years ago
- Structured Pruning Adapters in PyTorch☆19Updated 2 years ago
- Curse-of-memory phenomenon of RNNs in sequence modelling☆19Updated 6 months ago
- [NeurIPS'24] Official PyTorch implementation for paper "Knowledge Composition using Task Vectors with Learned Anisotropic Scaling"☆26Updated 9 months ago
- Recycling diverse models☆46Updated 2 years ago
- Code for the PAPA paper☆27Updated 3 years ago
- ☆81Updated last year
- The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …☆16Updated 5 months ago
- Official Code for ICLR 2024 Paper: Non-negative Contrastive Learning☆46Updated last year
- [NeurIPS 2022] Your Transformer May Not be as Powerful as You Expect (official implementation)☆34Updated 2 years ago
- HGRN2: Gated Linear RNNs with State Expansion☆55Updated last year
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆78Updated 2 years ago
- Official repo of Progressive Data Expansion: data, code and evaluation☆29Updated 2 years ago
- ☆42Updated 5 years ago
- ☆38Updated last year
- Pytorch implementation for "The Surprising Positive Knowledge Transfer in Continual 3D Object Shape Reconstruction"☆33Updated 3 years ago
- ☆16Updated 2 years ago
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆39Updated 3 years ago
- Code and results accompanying our paper titled RLSbench: Domain Adaptation under Relaxed Label Shift☆35Updated 2 years ago
- [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…☆56Updated 2 years ago
- Latest Weight Averaging (NeurIPS HITY 2022)☆31Updated 2 years ago
- A project to improve out-of-distribution detection (open set recognition) and uncertainty estimation by changing a few lines of code in y…☆44Updated 3 years ago
- Code to reproduce the results for Compositional Attention☆59Updated 3 years ago
- A project to add scalable state-of-the-art out-of-distribution detection (open set recognition) support by changing two lines of code! Pe…☆79Updated 3 years ago
- DiWA: Diverse Weight Averaging for Out-of-Distribution Generalization☆31Updated 2 years ago
- Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]☆60Updated 11 months ago
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆24Updated 2 years ago
- Code for "Theoretical Foundations of Deep Selective State-Space Models" (NeurIPS 2024)☆15Updated 10 months ago
- Why Do We Need Weight Decay in Modern Deep Learning? [NeurIPS 2024]☆68Updated last year
- LISA for ICML 2022☆51Updated 2 years ago