minhtannguyen / transformer-mgk
This is the public github for our paper "Transformer with a Mixture of Gaussian Keys"
☆26Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for transformer-mgk
- Code for the PAPA paper☆27Updated 2 years ago
- Curse-of-memory phenomenon of RNNs in sequence modelling☆19Updated this week
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆59Updated 2 years ago
- ☆21Updated last year
- HGRN2: Gated Linear RNNs with State Expansion☆49Updated 3 months ago
- Distributional Sliced-Wasserstein distance code☆48Updated 4 months ago
- The official repository for our paper "The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns …☆16Updated last year
- ☆14Updated 4 years ago
- [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…☆44Updated last year
- ☆21Updated last year
- Deep Learning & Information Bottleneck☆50Updated last year
- Official Code for ICML 2023 Data-Efficient Contrastive Self-supervised Learning☆30Updated 7 months ago
- Recycling diverse models☆44Updated last year
- [ICLR 2021] "Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning" by Tianlong Chen*, Zhenyu Zhang*, Sijia Liu, S…☆23Updated 2 years ago
- A project to add scalable state-of-the-art out-of-distribution detection (open set recognition) support by changing two lines of code! Pe…☆74Updated 2 years ago
- ☆81Updated 3 months ago
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆36Updated 2 years ago
- [ICLR 2023] “ Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Better Representations”, Ziyu Jian…☆23Updated last year
- A project to improve out-of-distribution detection (open set recognition) and uncertainty estimation by changing a few lines of code in y…☆45Updated 2 years ago
- [ICML2023] Instant Soup Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models. Ajay Jaiswal, Shiwei Liu, Ti…☆11Updated 11 months ago
- Mixture of Attention Heads☆39Updated 2 years ago
- Code for the paper "Data Feedback Loops: Model-driven Amplification of Dataset Biases"☆15Updated 2 years ago
- DiWA: Diverse Weight Averaging for Out-of-Distribution Generalization☆28Updated last year
- Structured Pruning Adapters in PyTorch☆15Updated last year
- Code to reproduce the results for Compositional Attention☆60Updated 2 years ago
- Official Code for ICLR 2024 Paper: Non-negative Contrastive Learning☆37Updated 7 months ago
- Pytorch implementation for "The Surprising Positive Knowledge Transfer in Continual 3D Object Shape Reconstruction"☆33Updated 2 years ago
- Code for "Are “Hierarchical” Visual Representations Hierarchical?" in NeurIPS Workshop for Symmetry and Geometry in Neural Representation…☆19Updated last year
- LibMoE: A LIBRARY FOR COMPREHENSIVE BENCHMARKING MIXTURE OF EXPERTS IN LARGE LANGUAGE MODELS☆30Updated last week
- LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters☆23Updated 2 weeks ago