nebius / kvaxLinks
A FlashAttention implementation for JAX with support for efficient document mask computation and context parallelism.
☆144Updated 6 months ago
Alternatives and similar repositories for kvax
Users that are interested in kvax are comparing it to the libraries listed below
Sorting:
- Minimal yet performant LLM examples in pure JAX☆181Updated 2 weeks ago
- A simple library for scaling up JAX programs☆143Updated 11 months ago
- seqax = sequence modeling + JAX☆167Updated 2 months ago
- jax-triton contains integrations between JAX and OpenAI Triton☆426Updated this week
- JAX-Toolbox☆348Updated this week
- 🧱 Modula software package☆282Updated last month
- ☆282Updated last year
- Dion optimizer algorithm☆361Updated last week
- JAX bindings for Flash Attention v2☆95Updated last month
- A zero-to-one guide on scaling modern transformers with n-dimensional parallelism.☆100Updated 2 weeks ago
- Write a fast kernel and run it on Discord. See how you compare against the best!☆58Updated 2 weeks ago
- Implementation of Diffusion Transformer (DiT) in JAX☆292Updated last year
- ☆67Updated 10 months ago
- Distributed pretraining of large language models (LLMs) on cloud TPU slices, with Jax and Equinox.☆24Updated last year
- Accelerated First Order Parallel Associative Scan☆189Updated last year
- Minimal, lightweight JAX implementations of popular models.☆110Updated this week
- An implementation of PSGD Kron second-order optimizer for PyTorch☆95Updated 2 months ago
- FlashRNN - Fast RNN Kernels with I/O Awareness☆98Updated 4 months ago
- Attention Kernels for Symmetric Power Transformers☆120Updated 2 weeks ago
- DeMo: Decoupled Momentum Optimization☆192Updated 10 months ago
- Minimal but scalable implementation of large language models in JAX☆35Updated last month
- Experiment of using Tangent to autodiff triton☆80Updated last year
- supporting pytorch FSDP for optimizers☆83Updated 10 months ago
- A Jax-based library for building transformers, includes implementations of GPT, Gemma, LlaMa, Mixtral, Whisper, SWin, ViT and more.☆293Updated last year
- Einsum-like high-level array sharding API for JAX☆35Updated last year
- Named Tensors for Legible Deep Learning in JAX☆207Updated last week
- Efficient optimizers☆269Updated this week
- Modular, scalable library to train ML models☆166Updated this week
- JMP is a Mixed Precision library for JAX.☆207Updated 8 months ago
- A library for unit scaling in PyTorch☆130Updated 3 months ago