PiotrNawrot / dynamic-pooling
Efficient Transformers with Dynamic Token Pooling
☆55Updated last year
Alternatives and similar repositories for dynamic-pooling:
Users that are interested in dynamic-pooling are comparing it to the libraries listed below
- Evaluation pipeline for the BabyLM Challenge 2023.☆73Updated last year
- DiffusER: Discrete Diffusion via Edit-based Reconstruction (Reid, Hellendoorn & Neubig, 2022)☆54Updated last year
- ☆93Updated last year
- Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023☆129Updated 8 months ago
- ☆64Updated 4 months ago
- Standalone Product Key Memory module in Pytorch - for augmenting Transformer models☆76Updated 5 months ago
- The original Backpack Language Model implementation, a fork of FlashAttention☆66Updated last year
- Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch☆97Updated last year
- This is the implementation of the paper AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning (https://arxiv.org/abs/2205.1…☆127Updated last year
- ☆11Updated 7 months ago
- No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models (ICLR 2022)☆30Updated 2 years ago
- ☆44Updated last year
- ☆80Updated 5 months ago
- DEMix Layers for Modular Language Modeling☆53Updated 3 years ago
- Teaching Models to Express Their Uncertainty in Words☆36Updated 2 years ago
- EMNLP 2021 - Frustratingly Simple Pretraining Alternatives to Masked Language Modeling☆31Updated 3 years ago
- ☆127Updated 2 years ago
- Official code and model checkpoints for our EMNLP 2022 paper "RankGen - Improving Text Generation with Large Ranking Models" (https://arx…☆136Updated last year
- A variant of Transformer-XL where the memory is updated not with a queue, but with attention☆47Updated 4 years ago
- Language modeling via stochastic processes. Oral @ ICLR 2022.☆137Updated last year
- ☆52Updated last year
- A library for parameter-efficient and composable transfer learning for NLP with sparse fine-tunings.☆71Updated 5 months ago
- [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…☆62Updated 8 months ago
- Skill-It! A Data-Driven Skills Framework for Understanding and Training Language Models☆42Updated last year
- ☆18Updated 2 years ago
- Randomized Positional Encodings Boost Length Generalization of Transformers☆79Updated 10 months ago
- The official repository for our paper "The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization".☆32Updated 3 years ago
- Code for Zero-Shot Tokenizer Transfer☆119Updated this week
- A Toolkit for Distributional Control of Generative Models☆70Updated last year