szhangtju / The-compression-of-Transformer
☆61Updated 3 years ago
Related projects: ⓘ
- Code for "Understanding and Improving Layer Normalization"☆44Updated 4 years ago
- ☆83Updated 4 years ago
- This package implements THOR: Transformer with Stochastic Experts.☆60Updated 2 years ago
- Compression of NMT transformer model with tensor methods☆46Updated 5 years ago
- Code for the paper "Are Sixteen Heads Really Better than One?"☆165Updated 4 years ago
- A simple module consistently outperforms self-attention and Transformer model on main NMT datasets with SoTA performance.☆86Updated last year
- [ICLR 2022] Code for paper "Exploring Extreme Parameter Compression for Pre-trained Language Models"(https://arxiv.org/abs/2205.10036)☆19Updated last year
- ☆32Updated 3 years ago
- Learning to Encode Position for Transformer with Continuous Dynamical Model☆59Updated 4 years ago
- For paper《Gaussian Transformer: A Lightweight Approach for Natural Language Inference》☆27Updated 4 years ago
- Language Model Baselines for PyTorch☆42Updated 4 years ago
- Parameter Efficient Transfer Learning with Diff Pruning☆70Updated 3 years ago
- Code for paper "Continual and Multi-Task Architecture Search (ACL 2019)"☆41Updated 5 years ago
- code for paper "Improving Sequence-to-Sequence Learning via Optimal Transport"☆69Updated 5 years ago
- code for Explicit Sparse Transformer☆57Updated last year
- ☆15Updated 3 years ago
- Pytorch library for factorized L0-based pruning.☆42Updated 11 months ago
- [EMNLP'19] Summary for Transformer Understanding☆51Updated 4 years ago
- FlatNCE: A Novel Contrastive Representation Learning Objective☆83Updated 2 years ago
- Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch☆70Updated 4 years ago
- ☆25Updated 2 years ago
- Codes for "Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View"☆146Updated 5 years ago
- ICLR2019, Multilingual Neural Machine Translation with Knowledge Distillation☆70Updated 3 years ago
- This repo provides the code for the ACL 2020 paper "Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEnco…☆52Updated 3 years ago
- EMNLP 2018: Multi-Head Attention with Disagreement Regularization; NAACL 2019: Information Aggregation for Multi-Head Attention with Rout…☆19Updated 3 years ago
- Implementation of Mogrifier LSTM in PyTorch☆35Updated 4 years ago
- ☆20Updated 4 years ago
- A PyTorch implementation of the paper - "Synthesizer: Rethinking Self-Attention in Transformer Models"☆70Updated last year
- Source code for "Efficient Training of BERT by Progressively Stacking"☆111Updated 5 years ago
- Variational Transformers for Diverse Response Generation☆82Updated last month