deep-spin / infinite-former
☆64Updated 6 months ago
Alternatives and similar repositories for infinite-former:
Users that are interested in infinite-former are comparing it to the libraries listed below
- Randomized Positional Encodings Boost Length Generalization of Transformers☆79Updated last year
- ☆52Updated 8 months ago
- Efficient Transformers with Dynamic Token Pooling☆60Updated last year
- Evaluation pipeline for the BabyLM Challenge 2023.☆75Updated last year
- ☆47Updated last year
- The official repository for our paper "The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization".☆33Updated 3 years ago
- Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023☆130Updated 10 months ago
- ☆44Updated last year
- ☆95Updated last year
- Transformers at any scale☆41Updated last year
- [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…☆64Updated 10 months ago
- ☆30Updated last year
- Recurrent Memory Transformer☆149Updated last year
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"☆36Updated last year
- ☆19Updated 2 years ago
- [TMLR'23] Contrastive Search Is What You Need For Neural Text Generation☆119Updated 2 years ago
- Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"☆56Updated 2 years ago
- Retrieval as Attention☆83Updated 2 years ago
- The original Backpack Language Model implementation, a fork of FlashAttention☆66Updated last year
- My explorations into editing the knowledge and memories of an attention network☆34Updated 2 years ago
- Language modeling via stochastic processes. Oral @ ICLR 2022.☆137Updated last year
- ☆11Updated 9 months ago
- [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling☆36Updated last year
- Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…☆53Updated last year
- Language models scale reliably with over-training and on downstream tasks☆96Updated 11 months ago
- Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model☆42Updated last year
- Standalone Product Key Memory module in Pytorch - for augmenting Transformer models☆78Updated 7 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆76Updated last year
- ☆51Updated 9 months ago
- This repository contains the data and code for the paper "Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Pr…☆24Updated 2 years ago