Alternatives and similar repositories for GLMKD:

Users that are interested in GLMKD are comparing it to the libraries listed below

• yegcjs / mixinglaws
  ☆93Updated 3 months ago
• sail-sg / regmix
  🧬 RegMix: Data Mixture as Regression for Language Model Pre-training
  ☆97Updated 3 months ago
• princeton-nlp / QuRating
  [ICML 2024] Selecting High-Quality Data for Training Language Models
  ☆155Updated 6 months ago
• swj0419 / in-context-pretraining
  ☆47Updated 9 months ago
• DAMO-NLP-SG / CLEX
  [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
  ☆75Updated 10 months ago
• OpenLMLab / LongWanjuan
  Towards Systematic Measurement for Long Text Quality
  ☆31Updated 4 months ago
• princeton-nlp / CEPE
  [ACL 2024] Long-Context Language Modeling with Parallel Encodings
  ☆154Updated 7 months ago
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆62Updated last year
• mtbench101 / mt-bench-101
  [ACL 2024] MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues
  ☆61Updated 5 months ago
• cxcscmu / MATES
  Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]
  ☆57Updated 2 months ago
• Shwai-He / MEO
  The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":
  ☆34Updated 9 months ago
• ernie-research / Tool-Augmented-Reward-Model
  [ICLR'24 spotlight] Tool-Augmented Reward Modeling
  ☆44Updated 3 weeks ago
• FranxYao / FlanT5-CoT-Specialization
  Implementation of ICML 23 Paper: Specializing Smaller Language Models towards Multi-Step Reasoning.
  ☆128Updated last year
• OhadRubin / EPR
  ☆60Updated 2 years ago
• tianyi-lab / Superfiltering
  [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
  ☆137Updated 4 months ago
• qtli / GSM-Plus
  GSM-Plus: Data, Code, and Evaluation for Enhancing Robust Mathematical Reasoning in Math Word Problems.
  ☆55Updated 6 months ago
• SimiaoZuo / MoEBERT
  This PyTorch package implements MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation (NAACL 2022).
  ☆97Updated 2 years ago
• QwenLM / online_merging_optimizers
  Implementations of online merging optimizers proposed by Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment
  ☆70Updated 7 months ago
• swtheing / PF-PPO-RLHF
  ☆30Updated 4 months ago
• thunlp / MoEfication
  ☆118Updated 5 months ago
• ZeroYuHuang / Transformer-Patcher
  ☆28Updated last year
• songmzhang / DSKD
  Repo for the EMNLP'24 Paper "Dual-Space Knowledge Distillation for Large Language Models".
  ☆40Updated 2 months ago
• KwanWaiChung / M4LE
  Code for M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models
  ☆22Updated 5 months ago
• jzbjyb / ReAtt
  Retrieval as Attention
  ☆83Updated 2 years ago
• wangpf3 / consistent-CoT-distillation
  ☆39Updated last year
• chijames / KERPLE
  ☆18Updated 2 years ago
• ZitongYang / Synthetic_Continued_Pretraining
  Code implementation of synthetic continued pretraining
  ☆79Updated last week
• jongwooko / distillm
  Official PyTorch implementation of DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024)
  ☆171Updated 3 months ago
• ChaosCodes / ProPETL
  One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning
  ☆38Updated last year
• GeneZC / MiniMoE
  Code for ACL 2023 paper titled "Lifting the Curse of Capacity Gap in Distilling Language Models"
  ☆28Updated last year