vvvm23 / mamba-jax
Unofficial but Efficient Implementation of "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" in JAX
☆79Updated 9 months ago
Related projects ⓘ
Alternatives and complementary repositories for mamba-jax
- ☆29Updated this week
- ☆46Updated last month
- ☆29Updated 7 months ago
- ☆35Updated 7 months ago
- A MAD laboratory to improve AI architecture designs 🧪☆95Updated 6 months ago
- Pytorch-like dataloaders in JAX.☆59Updated last month
- ☆36Updated 10 months ago
- ☆53Updated 10 months ago
- Implementation of GateLoop Transformer in Pytorch and Jax☆86Updated 5 months ago
- Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.☆29Updated 3 weeks ago
- Scalable neural net training via automatic normalization in the modular norm.☆122Updated this week
- Parallelizing non-linear sequential models over the sequence length☆45Updated 3 weeks ago
- A simple library for scaling up JAX programs☆127Updated 3 weeks ago
- A State-Space Model with Rational Transfer Function Representation.☆70Updated 6 months ago
- Automatically take good care of your preemptible TPUs☆32Updated last year
- ☆129Updated last week
- Sequence Modeling with Structured State Spaces☆60Updated 2 years ago
- Lightning-like training API for JAX with Flax☆34Updated 6 months ago
- Engineering the state of RNN language models (Mamba, RWKV, etc.)☆32Updated 5 months ago
- LoRA for arbitrary JAX models and functions☆133Updated 8 months ago
- ☆50Updated 6 months ago
- Blog post☆16Updated 9 months ago
- Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch☆95Updated last year
- Sequence Modeling with Multiresolution Convolutional Memory (ICML 2023)☆120Updated last year
- This is a port of Mistral-7B model in JAX☆30Updated 4 months ago
- Triton Implementation of HyperAttention Algorithm☆46Updated 11 months ago
- An annotated implementation of the Hyena Hierarchy paper☆31Updated last year
- Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…☆49Updated last year
- ☆48Updated 9 months ago