Transformer with Untied Positional Encoding (TUPE). Code of paper "Rethinking Positional Encoding in Language Pre-training". Improve existing models like BERT.
☆250Nov 8, 2021Updated 4 years ago
Alternatives and similar repositories for TUPE
Users that are interested in TUPE are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆221Jun 8, 2020Updated 6 years ago
- Code for the Shortformer model, from the ACL 2021 paper by Ofir Press, Noah A. Smith and Mike Lewis.☆145Jul 26, 2021Updated 5 years ago
- Source code for "Efficient Training of BERT by Progressively Stacking"☆112Jul 3, 2019Updated 7 years ago
- ☆100Jul 7, 2020Updated 6 years ago
- DeLighT: Very Deep and Light-Weight Transformers☆469Oct 16, 2020Updated 6 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- An efficient implementation of the popular sequence models for text generation, summarization, and translation tasks. https://arxiv.org/p…☆432Aug 17, 2022Updated 4 years ago
- ☆20Feb 26, 2021Updated 5 years ago
- ☆84Nov 14, 2019Updated 6 years ago
- Official Pytorch Implementation of Length-Adaptive Transformer (ACL 2021)☆102Nov 2, 2020Updated 6 years ago
- MPNet: Masked and Permuted Pre-training for Language Understanding https://arxiv.org/pdf/2004.09297.pdf☆301Sep 11, 2021Updated 5 years ago
- ☆262Oct 4, 2022Updated 4 years ago
- The implementation of "Neural Machine Translation without Embeddings", NAACL 2021☆33Jun 9, 2021Updated 5 years ago
- Understanding the Difficulty of Training Transformers☆332May 31, 2022Updated 4 years ago
- ☆878May 24, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Tracking the progress in non-autoregressive generation (translation, transcription, etc.)☆298Mar 15, 2023Updated 3 years ago
- The codebase for "Group-wise Contrastive Learning for Neural Dialogue Generation" (Cai et al., Findings of EMNLP 2020)☆55Feb 24, 2021Updated 5 years ago
- Cascaded Text Generation with Markov Transformers☆130Mar 20, 2023Updated 3 years ago
- The implementation of DeBERTa☆2,263Sep 29, 2023Updated 3 years ago
- Code for using and evaluating SpanBERT.☆907Jul 25, 2023Updated 3 years ago
- Dependency-Based Self-Attention for Transformer NMT☆12Mar 27, 2024Updated 2 years ago
- Single Headed Attention RNN - "Stop thinking with your head"☆1,181Nov 27, 2021Updated 4 years ago
- Transformer training code for sequential tasks☆610Sep 14, 2021Updated 5 years ago
- Sinkhorn Transformer - Practical implementation of Sparse Sinkhorn Attention☆272Aug 10, 2021Updated 5 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆15May 26, 2021Updated 5 years ago
- A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering☆43Nov 8, 2020Updated 5 years ago
- codes and pre-trained models of paper "Segatron: Segment-aware Transformer for Language Modeling and Understanding"☆18Oct 25, 2022Updated 4 years ago
- [ACL2020] Unsupervised Opinion Summarization with Noising and Denoising☆21Jul 3, 2020Updated 6 years ago
- Language model with phrase induction☆14Jun 13, 2019Updated 7 years ago
- A python library for highly configurable transformers - easing model architecture search and experimentation.☆48Nov 30, 2021Updated 4 years ago
- Solution of KDD cup 2021☆11Jun 16, 2021Updated 5 years ago
- Code for ACL2020 "Jointly Masked Sequence-to-Sequence Model for Non-Autoregressive Neural Machine Translation"☆38Jun 24, 2020Updated 6 years ago
- A simple Transformer where the softmax has been replaced with normalization☆20Sep 11, 2020Updated 6 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- custom cuda kernel for {2, 3}d relative attention with pytorch wrapper☆43May 5, 2020Updated 6 years ago
- [ACL 2021] Learning Dense Representations of Phrases at Scale; EMNLP'2021: Phrase Retrieval Learns Passage Retrieval, Too https://arxiv.o…☆606Jun 15, 2022Updated 4 years ago
- Modified version of fairseq, including new implementations for criterions using reinforcement learning methods.☆11Aug 14, 2019Updated 7 years ago
- ☆70Oct 27, 2020Updated 6 years ago
- Code associated with the Don't Stop Pretraining ACL 2020 paper☆546Nov 15, 2021Updated 4 years ago
- Implementation of the Remixer Block from the Remixer paper, in Pytorch☆36Sep 27, 2021Updated 5 years ago
- Multi-Task Deep Neural Networks for Natural Language Understanding☆2,258Mar 7, 2024Updated 2 years ago