ethansmith2000 / TransformerExperiments

Related projects ⓘ

Alternatives and complementary repositories for TransformerExperiments

• cloneofsimo / zeroshampoo
  ☆31Updated 2 months ago
• EleutherAI / rnngineering
  Engineering the state of RNN language models (Mamba, RWKV, etc.)
  ☆32Updated 5 months ago
• crowsonkb / torch-dist-utils
  Utilities for PyTorch distributed
  ☆23Updated last year
• ClashLuke / HeavyBall
  Efficient optimizers
  ☆42Updated this week
• thecharlieblake / lovely-llama
  An implementation of the Llama architecture, to instruct and delight
  ☆21Updated 2 months ago
• cat-state / tinypar
  ☆20Updated last year
• ClashLuke / tpucare
  Automatically take good care of your preemptible TPUs
  ☆31Updated last year
• crowsonkb / LDLM
  Latent Diffusion Language Models
  ☆67Updated last year
• shikaiqiu / compute-better-spent
  ☆46Updated last month
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆39Updated 3 months ago
• epfml / DenseFormer
  ☆76Updated 6 months ago
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆46Updated 10 months ago
• maximzubkov / fft-scan
  Efficient PScan implementation in PyTorch
  ☆15Updated 10 months ago
• google-deepmind / spectral_ssm
  ☆27Updated 6 months ago
• RobertCsordas / moeut
  ☆61Updated 2 months ago
• ClashLuke / SOAP
  ☆18Updated this week
• cloneofsimo / min-fsdp
  ☆72Updated 4 months ago
• sjelassi / transformers_ssm_copy
  ☆24Updated 8 months ago
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆71Updated 9 months ago
• evanatyourservice / kron_torch
  An implementation of PSGD Kron second-order optimizer for PyTorch
  ☆15Updated this week
• EleutherAI / nanoGPT-mup
  The simplest, fastest repository for training/finetuning medium-sized GPTs.
  ☆83Updated last week
• dvruette / barrel-rec-pytorch
  ☆53Updated 9 months ago
• borisdayma / clip-jax
  Train vision models using JAX and 🤗 transformers
  ☆95Updated 2 weeks ago
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆66Updated this week
• lucidrains / gateloop-transformer
  Implementation of GateLoop Transformer in Pytorch and Jax
  ☆86Updated 4 months ago
• proger / nanokitchen
  Parallel Associative Scan for Language Models
  ☆18Updated 10 months ago
• glassroom / heinsen_attention
  Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
  ☆23Updated 5 months ago
• lucaslingle / mu_transformer
  Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.
  ☆29Updated last week
• lucidrains / self-reasoning-tokens-pytorch
  Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
  ☆53Updated 5 months ago