Alternatives and similar repositories for in-context-pretraining:

Users that are interested in in-context-pretraining are comparing it to the libraries listed below

• nayeon7lee / FactualityPrompt
  ☆85Updated 2 years ago
• yuzhaouoe / pretraining-data-packing
  [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training
  ☆20Updated 8 months ago
• qtli / GSM-Plus
  GSM-Plus: Data, Code, and Evaluation for Enhancing Robust Mathematical Reasoning in Math Word Problems.
  ☆59Updated 9 months ago
• hkust-nlp / felm
  Github repository for "FELM: Benchmarking Factuality Evaluation of Large Language Models" (NeurIPS 2023)
  ☆58Updated last year
• KwanWaiChung / M4LE
  Code for M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models
  ☆23Updated 8 months ago
• wzhouad / context-faithful-llm
  Code and data for paper "Context-faithful Prompting for Large Language Models".
  ☆39Updated 2 years ago
• nkandpa2 / long_tail_knowledge
  Repo for the paper "Large Language Models Struggle to Learn Long-Tail Knowledge"
  ☆76Updated 2 years ago
• sail-sg / symbolic-instruction-tuning
  The official repository for the paper "From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuning".
  ☆64Updated 2 years ago
• OpenLMLab / LongWanjuan
  Towards Systematic Measurement for Long Text Quality
  ☆34Updated 7 months ago
• yizhongw / llm-temporal-alignment
  Methods and evaluation for aligning language models temporally
  ☆29Updated last year
• shankarp8 / knowledge_distillation
  Repository for "Propagating Knowledge Updates to LMs Through Distillation" (NeurIPS 2023).
  ☆25Updated 8 months ago
• FranxYao / FlanT5-CoT-Specialization
  Implementation of ICML 23 Paper: Specializing Smaller Language Models towards Multi-Step Reasoning.
  ☆130Updated last year
• YuxiXie / SelfEval-Guided-Decoding
  ☆95Updated last year
• OhadRubin / EPR
  ☆61Updated 2 years ago
• jzbjyb / ReAtt
  Retrieval as Attention
  ☆83Updated 2 years ago
• princeton-nlp / QuRating
  [ICML 2024] Selecting High-Quality Data for Training Language Models
  ☆168Updated 10 months ago
• ZeroYuHuang / Transformer-Patcher
  ☆31Updated last year
• Zce1112zslx / IKE
  ☆41Updated last year
• HKUNLP / icl-ceil
  [ICML 2023] Code for our paper “Compositional Exemplars for In-context Learning”.
  ☆99Updated 2 years ago
• PrasannS / rlhf-length-biases
  ☆28Updated last year
• yegcjs / mixinglaws
  ☆98Updated 6 months ago
• swj0419 / kNN_prompt
  TBC
  ☆26Updated 2 years ago
• FreedomIntelligence / OVM
  ☆65Updated last year
• Nanami18 / Snowballed_Hallucination
  ☆44Updated 7 months ago
• PlusLabNLP / Active-IT
  Code for our EMNLP-2023 paper: "Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks"
  ☆24Updated last year
• OSU-NLP-Group / LLM-Knowledge-Conflict
  [ICLR'24 Spotlight] "Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts"
  ☆67Updated last year
• edenbiran / RippleEdits
  Evaluating the Ripple Effects of Knowledge Editing in Language Models
  ☆55Updated last year
• ChaosCodes / ProPETL
  One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning
  ☆39Updated last year
• cxcscmu / MATES
  Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]
  ☆64Updated 5 months ago
• chujiezheng / LLM-Extrapolation
  Official repository for paper "Weak-to-Strong Extrapolation Expedites Alignment"
  ☆74Updated 10 months ago