Kirill-Kravtsov / drophead-pytorch
An implementation of drophead regularization for pytorch transformers
β19Updated 3 years ago
Alternatives and similar repositories for drophead-pytorch:
Users that are interested in drophead-pytorch are comparing it to the libraries listed below
- Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorchβ45Updated 3 years ago
- Implementation of Mixout with PyTorchβ74Updated 2 years ago
- My 6thπ₯ place solution for Kaggle Shopee competition.β27Updated 2 years ago
- source code and pre-trained/fine-tuned checkpoint for NAACL 2021 paper LightningDOTβ73Updated 2 years ago
- β26Updated 3 years ago
- β44Updated 3 years ago
- Implementation of the retriever distillation procedure as outlined in the paper "Distilling Knowledge from Reader to Retriever"β32Updated 4 years ago
- This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron β¦β32Updated last year
- The official implementation of InterBERTβ11Updated 2 years ago
- Implementation of Online Label Smoothing in PyTorchβ94Updated 2 years ago
- Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012β49Updated 2 years ago
- Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorchβ118Updated 3 years ago
- Solution of Kaggle competition: Feedback Prize - Evaluating Student Writingβ17Updated 2 years ago
- Code for ACL 2023 Oral Paper: ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learningβ11Updated last month
- [FGVC9-CVPR 2022] The second place solution for 2nd eBay eProduct Visual Search Challenge.β26Updated 2 years ago
- Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorchβ70Updated 4 years ago
- PyTorch code for "Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention" (WACV 2023)β33Updated 2 years ago
- β12Updated 2 years ago
- Code for EMNLP 2020 paper CoDIRβ41Updated 2 years ago
- Implementation of N-Grammer, augmenting Transformers with latent n-grams, in Pytorchβ72Updated 2 years ago
- codebase for the SIMAT dataset and evaluationβ38Updated 3 years ago
- Official code for Group-Transformer (Scale down Transformer by Grouping Features for a Lightweight Character-level Language Model, COLINGβ¦β25Updated 4 years ago
- The implementation of multi-branch attentive Transformer (MAT).β33Updated 4 years ago
- Implementation for ACProp ( Momentum centering and asynchronous update for adaptive gradient methdos, NeurIPS 2021)β15Updated 3 years ago
- Implementation of Multistream Transformers in Pytorchβ53Updated 3 years ago
- ImageNet-12k subset of ImageNet-21k (fall11)β21Updated last year
- Source code repo for paper "TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation"β10Updated last year
- A dataset of crowdsourced ratings for machine-generated image captionsβ35Updated 5 years ago
- β29Updated 2 years ago
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)β60Updated 2 years ago