jopetty / word-problem
Experiments on the impact of depth in transformers and SSMs.
☆14Updated 2 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for word-problem
- ☆46Updated last month
- ☆53Updated 10 months ago
- ☆50Updated 6 months ago
- A MAD laboratory to improve AI architecture designs 🧪☆95Updated 6 months ago
- ☆29Updated this week
- Scalable neural net training via automatic normalization in the modular norm.☆122Updated this week
- Unofficial but Efficient Implementation of "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" in JAX☆79Updated 10 months ago
- ☆29Updated 7 months ago
- Evaluating the Mamba architecture on the Othello game☆43Updated 6 months ago
- LoRA for arbitrary JAX models and functions☆133Updated 8 months ago
- ☆129Updated last week
- Efficient PScan implementation in PyTorch☆15Updated 10 months ago
- Implementation of GateLoop Transformer in Pytorch and Jax☆86Updated 5 months ago
- A State-Space Model with Rational Transfer Function Representation.☆70Updated 6 months ago
- GoldFinch and other hybrid transformer components☆40Updated 4 months ago
- JAX bindings for Flash Attention v2☆80Updated 4 months ago
- ☆16Updated 3 months ago
- Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.☆29Updated 3 weeks ago
- ☆73Updated 4 months ago
- Minimal but scalable implementation of large language models in JAX☆26Updated 3 weeks ago
- Universal Neurons in GPT2 Language Models☆27Updated 5 months ago
- Accelerated First Order Parallel Associative Scan☆164Updated 3 months ago
- Implementation of PSGD optimizer in JAX☆19Updated 2 weeks ago
- If it quacks like a tensor...☆52Updated last week
- minGPT in JAX☆46Updated 2 years ago
- Latent Diffusion Language Models☆67Updated last year
- Mixture of A Million Experts☆32Updated 3 months ago
- ☆18Updated last month
- Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam☆68Updated 3 months ago
- ☆36Updated 10 months ago