warner-benjamin / optimi

Related projects ⓘ

Alternatives and complementary repositories for optimi

• cat-state / tinypar
  ☆20Updated last year
• euclaise / supertrainer2000
  ☆49Updated 8 months ago
• kyleliang919 / Long-context-transformers
  Exploring finetuning public checkpoints on filter 8K sequences on Pile
  ☆115Updated last year
• Rallio67 / language-model-agents
  Experiments with generating opensource language model assistants
  ☆97Updated last year
• catie-aq / flashT5
  A fast implementation of T5/UL2 in PyTorch using Flash Attention
  ☆71Updated last month
• cloneofsimo / min-fsdp
  ☆73Updated 4 months ago
• EleutherAI / nanoGPT-mup
  The simplest, fastest repository for training/finetuning medium-sized GPTs.
  ☆84Updated this week
• mgmalek / efficient_cross_entropy
  ☆77Updated 5 months ago
• lessw2020 / transformer_central
  Various transformers for FSDP research
  ☆33Updated 2 years ago
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆45Updated last year
• EleutherAI / improved-t5
  Experiments for efforts to train a new and improved t5
  ☆76Updated 7 months ago
• epfml / DenseFormer
  ☆77Updated 7 months ago
• kjslag / spacebyte
  A byte-level decoder architecture that matches the performance of tokenized Transformers.
  ☆59Updated 6 months ago
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆45Updated 2 months ago
• KellerJordan / Muon
  Muon optimizer for neural networks: >30% extra sample efficiency, <3% wallclock overhead
  ☆121Updated this week
• HomebrewNLP / Olmax
  HomebrewNLP in JAX flavour for maintable TPU-Training
  ☆46Updated 10 months ago
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated last year
• jxmorris12 / cde
  code for training & evaluating Contextual Document Embedding models
  ☆119Updated this week
• ltgoslo / gpt-bert
  Official implementation of "GPT or BERT: why not both?"
  ☆37Updated last week
• HazyResearch / zoology
  Understand and test language model architectures on synthetic tasks.
  ☆163Updated 6 months ago
• microsoft / mutransformers
  some common Huggingface transformers in maximal update parametrization (µP)
  ☆77Updated 2 years ago
• Aleph-Alpha / trigrams
  ☆43Updated 2 months ago
• huggingface / chug
  Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.
  ☆152Updated 7 months ago
• AnswerDotAI / bert24
  ☆66Updated this week
• arcee-ai / DAM
  ☆41Updated 2 weeks ago
• huggingface / olm-training
  Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.
  ☆92Updated last year
• epfml / schedules-and-scaling
  ☆53Updated 3 weeks ago
• cloneofsimo / min-max-gpt
  Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training
  ☆113Updated 7 months ago
• Zyphra / Zyda_processing
  ☆27Updated 5 months ago