Alternatives and similar repositories for staged-training:

Users that are interested in staged-training are comparing it to the libraries listed below

• kernelmachine / demix
  DEMix Layers for Modular Language Modeling
  ☆53Updated 3 years ago
• frankxu2004 / knnlm-why
  Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"
  ☆56Updated 2 years ago
• violet-zct / swarm-distillation-zero-shot
  ☆22Updated 2 years ago
• Shark-NLP / CAB
  ☆30Updated last year
• cliang1453 / SAGE
  No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models (ICLR 2022)
  ☆30Updated 2 years ago
• ThomasScialom / T0_continual_learning
  Adding new tasks to T0 without catastrophic forgetting
  ☆32Updated 2 years ago
• jungokasai / deep-shallow
  ☆43Updated 4 years ago
• jxhe / efficient-knnlm
  Pytorch implementation of paper "Efficient Nearest Neighbor Language Models" (EMNLP 2021)
  ☆71Updated 3 years ago
• yxuansu / Contrastive_Search_versus_Contrastive_Decoding
  An Empirical Study On Contrastive Search And Contrastive Decoding For Open-ended Text Generation
  ☆26Updated 7 months ago
• McGill-NLP / polytropon
  ☆52Updated last year
• thunlp / Knowledge-Inheritance
  Source code for paper: Knowledge Inheritance for Pre-trained Language Models
  ☆38Updated 2 years ago
• kyegomez / Reka-Torch
  Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch
  ☆28Updated this week
• suzgunmirac / crowd-sampling
  Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding
  ☆18Updated 2 years ago
• nkandpa2 / long_tail_knowledge
  Repo for the paper "Large Language Models Struggle to Learn Long-Tail Knowledge"
  ☆74Updated last year
• ghrua / NgramRes
  ☆21Updated 2 years ago
• INK-USC / NExT
  Source Code for paper "Learning from Explanations with Neural Execution Tree", ICLR 2020
  ☆18Updated 3 years ago
• RUCAIBox / ELMER
  This repository is the official implementation of our EMNLP 2022 paper ELMER: A Non-Autoregressive Pre-trained Language Model for Efficie…
  ☆27Updated 2 years ago
• ghomasHudson / muld
  The Multitask Long Document Benchmark
  ☆38Updated 2 years ago
• intersun / CoDIR
  Code for EMNLP 2020 paper CoDIR
  ☆41Updated 2 years ago
• fuzihaofzh / repetition-problem-nlg
  Code for the paper "A Theoretical Analysis of the Repetition Problem in Text Generation" in AAAI 2021.
  ☆51Updated 2 years ago
• machelreid / m2d2
  M2D2: A Massively Multi-domain Language Modeling Dataset (EMNLP 2022) by Machel Reid, Victor Zhong, Suchin Gururangan, Luke Zettlemoyer
  ☆55Updated 2 years ago
• jungokasai / twist_decoding
  ☆28Updated 2 years ago
• INK-USC / ReCross
  ReCross: Unsupervised Cross-Task Generalization via Retrieval Augmentation
  ☆24Updated 2 years ago
• facebookresearch / UNLU
  Code for the paper "UnNatural Language Inference" to appear at ACL 2021 (Long Paper)
  ☆36Updated 3 years ago
• llyx97 / TAMT
  [NAACL 2022] "Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training", Yuanxin Liu, Fandong Meng, Zheng Lin, Pe…
  ☆15Updated 2 years ago
• gmftbyGMFTBY / Rep-Dropout
  [NeurIPS 2023] Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective
  ☆30Updated last year
• ekinakyurek / influence
  Code for "Tracing Knowledge in Language Models Back to the Training Data"
  ☆37Updated 2 years ago
• felixzli / synthetic_pretraining
  ☆39Updated 2 years ago
• GeneZC / MiniMoE
  Code for ACL 2023 paper titled "Lifting the Curse of Capacity Gap in Distilling Language Models"
  ☆28Updated last year
• tigerchen52 / LOVE
  ACL22 paper: Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Cost
  ☆40Updated last year