evanatyourservice / psgd_jax

Related projects ⓘ

Alternatives and complementary repositories for psgd_jax

• GallagherCommaJack / modulax
  ☆16Updated 3 months ago
• dvruette / barrel-rec-pytorch
  ☆53Updated 10 months ago
• ClashLuke / tpucare
  Automatically take good care of your preemptible TPUs
  ☆32Updated last year
• nikhilvyas / SOAP
  ☆129Updated last week
• cloneofsimo / min-fsdp
  ☆73Updated 4 months ago
• modula-systems / modula
  Scalable neural net training via automatic normalization in the modular norm.
  ☆122Updated this week
• ClashLuke / HeavyBall
  Efficient optimizers
  ☆87Updated this week
• MatX-inc / seqax
  seqax = sequence modeling + JAX
  ☆134Updated 4 months ago
• lixilinx / psgd_torch
  Pytorch implementation of preconditioned stochastic gradient descent (affine group preconditioner, low-rank approximation preconditioner …
  ☆128Updated last month
• davisyoshida / lorax
  LoRA for arbitrary JAX models and functions
  ☆133Updated 8 months ago
• young-geng / scalax
  A simple library for scaling up JAX programs
  ☆127Updated 3 weeks ago
• apple / ml-ademamix
  ☆48Updated last week
• cloneofsimo / scaling-guide
  WIP
  ☆89Updated 3 months ago
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆72Updated 10 months ago
• proger / accelerated-scan
  Accelerated First Order Parallel Associative Scan
  ☆164Updated 3 months ago
• berlino / seq_icl
  ☆50Updated 6 months ago
• cloneofsimo / min-max-gpt
  Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training
  ☆113Updated 7 months ago
• lucaslingle / mu_transformer
  Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.
  ☆29Updated 3 weeks ago
• evanatyourservice / kron_torch
  An implementation of PSGD Kron second-order optimizer for PyTorch
  ☆16Updated this week
• thecharlieblake / lovely-llama
  An implementation of the Llama architecture, to instruct and delight
  ☆21Updated 3 months ago
• ethansmith2000 / TransformerExperiments
  ☆18Updated last month
• cloneofsimo / ezmup
  Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam
  ☆68Updated 3 months ago
• srush / Tensor-Puzzles-Penzai
  ☆19Updated 7 months ago
• opooladz / Preconditioned-Stochastic-Gradient-Descent
  A repo based on XiLin Li's PSGD repo that extends some of the experiments.
  ☆14Updated last month
• KellerJordan / Muon
  Muon optimizer for neural networks: >30% extra sample efficiency, <3% wallclock overhead
  ☆121Updated this week
• google-deepmind / nanodo
  ☆198Updated 4 months ago
• Artur-Galstyan / statedict2pytree
  ☆40Updated 4 months ago
• athms / mad-lab
  A MAD laboratory to improve AI architecture designs 🧪
  ☆95Updated 6 months ago
• graphcore-research / unit-scaling
  A library for unit scaling in PyTorch
  ☆105Updated 2 weeks ago
• kvfrans / jax-flow
  Flow-matching algorithms in JAX
  ☆77Updated 3 months ago