10-zin / Synthesizer
A PyTorch implementation of the paper - "Synthesizer: Rethinking Self-Attention in Transformer Models"
☆71Updated last year
Related projects ⓘ
Alternatives and complementary repositories for Synthesizer
- Code for EMNLP 2020 paper CoDIR☆41Updated 2 years ago
- Implementation of the retriever distillation procedure as outlined in the paper "Distilling Knowledge from Reader to Retriever"☆32Updated 3 years ago
- Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch☆70Updated 4 years ago
- ☆22Updated 3 years ago
- Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch☆45Updated 3 years ago
- Implementation of Mixout with PyTorch☆74Updated last year
- DisCo Transformer for Non-autoregressive MT☆78Updated 2 years ago
- Axial Positional Embedding for Pytorch☆60Updated 3 years ago
- Code for "Understanding and Improving Layer Normalization"☆46Updated 4 years ago
- (ACL-IJCNLP 2021) Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models.☆21Updated 2 years ago
- The implementation of multi-branch attentive Transformer (MAT).☆33Updated 4 years ago
- Code for the ACL2020 paper Character-Level Translation with Self-Attention☆32Updated 4 years ago
- ☆20Updated 5 years ago
- Code for Multi-Head Attention: Collaborate Instead of Concatenate☆151Updated last year
- This repo provides the code for the ACL 2020 paper "Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEnco…☆52Updated 4 years ago
- ☆32Updated 3 years ago
- Cascaded Text Generation with Markov Transformers☆128Updated last year
- Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data☆56Updated 3 years ago
- FlatNCE: A Novel Contrastive Representation Learning Objective☆87Updated 3 years ago
- Official PyTorch implementation of Time-aware Large Kernel (TaLK) Convolutions (ICML 2020)☆29Updated 3 years ago
- ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation☆24Updated 4 years ago
- ☆63Updated 2 years ago
- Code for the paper "Query-Key Normalization for Transformers"☆35Updated 3 years ago
- 基于Transformer的单模型、多尺度的VAE模型☆53Updated 3 years ago
- This repository contains the code for running the character-level Sandwich Transformers from our ACL 2020 paper on Improving Transformer …☆55Updated 3 years ago
- PyTorch implementation of Pay Attention to MLPs☆39Updated 3 years ago
- Curriculum Learning related papers and materials☆54Updated 4 years ago
- Code for the paper "Adaptive Transformers for Learning Multimodal Representations" (ACL SRW 2020)☆42Updated 2 years ago
- Repository containing code for the paper "Meta-Learning with Sparse Experience Replay for Lifelong Language Learning".☆21Updated last year
- ICLR2019, Multilingual Neural Machine Translation with Knowledge Distillation☆70Updated 4 years ago