buttercutter / Mamba_SSMLinks
A simple implementation of [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752)
☆22Updated last year
Alternatives and similar repositories for Mamba_SSM
Users that are interested in Mamba_SSM are comparing it to the libraries listed below
Sorting:
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆49Updated 2 years ago
- Here we will test various linear attention designs.☆62Updated last year
- MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248☆57Updated last year
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- Experimental scripts for researching data adaptive learning rate scheduling.☆22Updated last year
- Official code for the paper "Image generation with shortest path diffusion" accepted at ICML 2023.☆23Updated 2 years ago
- Implementation of Infini-Transformer in Pytorch☆111Updated 8 months ago
- HGRN2: Gated Linear RNNs with State Expansion☆54Updated last year
- Implementation of 2-simplicial attention proposed by Clift et al. (2019) and the recent attempt to make practical in Fast and Simplex, Ro…☆46Updated 3 weeks ago
- CUDA implementation of autoregressive linear attention, with all the latest research findings☆44Updated 2 years ago
- Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts☆121Updated 11 months ago
- ☆32Updated last year
- ☆32Updated last year
- ☆23Updated last year
- Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.☆70Updated last month
- Official code for the paper "Attention as a Hypernetwork"☆42Updated last year
- ResiDual: Transformer with Dual Residual Connections, https://arxiv.org/abs/2304.14802☆95Updated 2 years ago
- ☆106Updated last year
- Xmixers: A collection of SOTA efficient token/channel mixers☆15Updated 3 weeks ago
- Explorations into adversarial losses on top of autoregressive loss for language modeling☆37Updated 7 months ago
- Linear Attention Sequence Parallelism (LASP)☆86Updated last year
- [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…☆102Updated last year
- ☆29Updated 2 years ago
- This is a PyTorch implementation of the paperViP A Differentially Private Foundation Model for Computer Vision☆36Updated 2 years ago
- Triton implement of bi-directional (non-causal) linear attention☆55Updated 7 months ago
- A repository for DenseSSMs☆88Updated last year
- ImageNet-12k subset of ImageNet-21k (fall11)☆21Updated 2 years ago
- Hacks for PyTorch☆19Updated 2 years ago
- Simple notebooks to learn diffusion models on toy datasets☆17Updated 2 years ago
- [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning☆132Updated last week