epfml / schedules-and-scaling

Related projects ⓘ

Alternatives and complementary repositories for schedules-and-scaling

• RobertCsordas / moeut
  ☆61Updated 2 months ago
• mlfoundations / scaling
  Language models scale reliably with over-training and on downstream tasks
  ☆94Updated 7 months ago
• berlino / seq_icl
  ☆50Updated 5 months ago
• ScalingIntelligence / large_language_monkeys
  ☆50Updated last month
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆61Updated 7 months ago
• Zyphra / Zyda_processing
  ☆26Updated 4 months ago
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆46Updated 10 months ago
• wang-kee / LiNeS
  ☆20Updated this week
• sjelassi / transformers_ssm_copy
  ☆24Updated 8 months ago
• young-geng / mintext
  Minimal but scalable implementation of large language models in JAX
  ☆25Updated last week
• EleutherAI / nanoGPT-mup
  The simplest, fastest repository for training/finetuning medium-sized GPTs.
  ☆83Updated last week
• shikaiqiu / compute-better-spent
  ☆46Updated last month
• activatedgeek / calibration-tuning
  ☆34Updated 3 months ago
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆40Updated last month
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆24Updated 6 months ago
• allenai / easy-to-hard-generalization
  Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"
  ☆44Updated 9 months ago
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆36Updated 11 months ago
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆32Updated last week
• epfml / llm-baselines
  NanoGPT-like codebase for LLM training
  ☆73Updated this week
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆39Updated 3 months ago
• RodkinIvan / associative-recurrent-memory-transformer
  [ICML 24 NGSM workshop] Associative Recurrent Memory Transformer implementation and scripts for training and evaluating
  ☆29Updated this week
• lucidrains / pause-transformer
  Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…
  ☆49Updated last year
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆18Updated last month
• JonasGeiping / linear_cross_entropy_loss
  A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.
  ☆54Updated 3 months ago
• mgmalek / efficient_cross_entropy
  ☆76Updated 5 months ago
• athms / mad-lab
  A MAD laboratory to improve AI architecture designs 🧪
  ☆95Updated 6 months ago
• wellecks / lm-evaluation-harness
  A framework for few-shot evaluation of autoregressive language models.
  ☆23Updated 10 months ago
• ml-jku / EVA
  One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation
  ☆29Updated 3 weeks ago
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆67Updated last month