kingoflolz / swarm-jax
Swarm training framework using Haiku + JAX + Ray for layer parallel transformer language models on unreliable, heterogeneous nodes
☆236Updated last year
Related projects: ⓘ
- Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)☆184Updated 2 years ago
- Train very large language models in Jax.☆191Updated 10 months ago
- Named tensors with first-class dimensions for PyTorch☆321Updated last year
- Python Research Framework☆107Updated last year
- ☆172Updated last week
- ☆322Updated 5 months ago
- ☆56Updated 2 years ago
- PIX is an image processing library in JAX, for JAX.☆379Updated 2 months ago
- jax-triton contains integrations between JAX and OpenAI Triton☆328Updated this week
- Implementation of Flash Attention in Jax☆188Updated 6 months ago
- Babysit your preemptible TPUs☆84Updated last year
- ☆149Updated 9 months ago
- JMP is a Mixed Precision library for JAX.☆183Updated 3 months ago
- Inference code for LLaMA models in JAX☆108Updated 3 months ago
- A pure-functional implementation of a machine learning transformer model in Python/JAX☆173Updated 2 years ago
- ☆64Updated 2 years ago
- JAX Synergistic Memory Inspector☆161Updated 2 months ago
- ☆156Updated 4 years ago
- ☆85Updated this week
- Pretrained deep learning models for Jax/Flax: StyleGAN2, GPT2, VGG, ResNet, etc.☆234Updated last year
- CLU lets you write beautiful training loops in JAX.☆318Updated 3 weeks ago
- JAX implementation of the Llama 2 model☆205Updated 7 months ago
- Implementation of a Transformer, but completely in Triton☆242Updated 2 years ago
- For optimization algorithm research and development.☆240Updated last week
- A Pytree Module system for Deep Learning in JAX☆215Updated last year
- A simple library for scaling up JAX programs☆116Updated last month
- ☆247Updated this week
- DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.☆163Updated 4 months ago
- Automatic gradient descent☆206Updated last year
- A library for distributed ML training with PyTorch☆365Updated last year