idiap / hypermixingLinks
PyTorch implementation for HyperMixing, a linear-time token-mixing technique used in HyperMixer architecture
☆25Updated 2 years ago
Alternatives and similar repositories for hypermixing
Users that are interested in hypermixing are comparing it to the libraries listed below
Sorting:
- Implementations of various linear RNN layers using pytorch and triton☆54Updated 2 years ago
- Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"☆56Updated 3 months ago
- Pytorch implementation of Simplified Structured State-Spaces for Sequence Modeling (S5)☆82Updated last year
- ☆26Updated last year
- Torch implementation of Soft-DTW, supports CUDA.☆48Updated 2 weeks ago
- Implementation of the proposed minGRU in Pytorch☆319Updated 2 months ago
- Sequence Modeling with Structured State Spaces☆67Updated 3 years ago
- [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…☆66Updated last year
- Griffin MQA + Hawk Linear RNN Hybrid☆88Updated last year
- ☆29Updated last year
- ☆163Updated 3 years ago
- Randomized Positional Encodings Boost Length Generalization of Transformers☆82Updated last year
- ☆28Updated last year
- an implementation of FAdam (Fisher Adam) in PyTorch☆50Updated 7 months ago
- This is the official repository of the papers "Parameter-Efficient Transfer Learning of Audio Spectrogram Transformers" and "Efficient Fi…☆38Updated last year
- Implementation of GateLoop Transformer in Pytorch and Jax☆92Updated last year
- ☆45Updated 7 years ago
- Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch☆135Updated 3 months ago
- The accompanying code for "Simplifying and Understanding State Space Models with Diagonal Linear RNNs" (Ankit Gupta, Harsh Mehta, Jonatha…☆23Updated 3 years ago
- Training RNNs as Fast as CNNs (https://arxiv.org/abs/1709.02755)☆38Updated 4 years ago
- Pytorch Reimplementation of DiffWave Vocoder: a high quality, fast, and small neural vocoder.☆91Updated 4 years ago
- [ICLR 2023] Official implementation of Transnormer in our ICLR 2023 paper - Toeplitz Neural Network for Sequence Modeling☆81Updated last year
- A State-Space Model with Rational Transfer Function Representation.☆83Updated last year
- [SLT'24] The official implementation of SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model☆134Updated 3 months ago
- Inspired by "Neural Networks Fail to Learn Periodic Functions and How to Fix It"☆73Updated 7 months ago
- A neural network layer API and library for sequence modeling, designed for easy creation of sequence models that can be executed layerwis…☆50Updated this week
- Experimental playground for benchmarking language model (LM) architectures, layers, and tricks on smaller datasets. Designed for flexible…☆98Updated 2 weeks ago
- Official Implementation of the work "Audio Mamba: Bidirectional State Space Model for Audio Representation Learning"☆167Updated last year
- Jupyter Notebook running Mamba speech synthesis example on Determined AI. Based on https://2084.substack.com/p/2084-marcrandbot-speech-sy…☆23Updated 2 years ago
- Parallelizing non-linear sequential models over the sequence length☆56Updated 7 months ago