yuzhaouoe / pretraining-data-packing
[ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training
☆18Updated 3 months ago
Related projects ⓘ
Alternatives and complementary repositories for pretraining-data-packing
- ☆39Updated 7 months ago
- "FiD-ICL: A Fusion-in-Decoder Approach for Efficient In-Context Learning" (ACL 2023)☆13Updated last year
- [NeurIPS 2023] Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective☆30Updated last year
- ☆40Updated 11 months ago
- Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"☆56Updated last year
- Code and data for paper "Context-faithful Prompting for Large Language Models".☆39Updated last year
- Repository for "Propagating Knowledge Updates to LMs Through Distillation" (NeurIPS 2023).☆24Updated 2 months ago
- This repository contains the dataset and code for "WiCE: Real-World Entailment for Claims in Wikipedia" in EMNLP 2023.☆40Updated 11 months ago
- Code for "Tracing Knowledge in Language Models Back to the Training Data"☆35Updated last year
- GSM-Plus: Data, Code, and Evaluation for Enhancing Robust Mathematical Reasoning in Math Word Problems.☆47Updated 4 months ago
- An Empirical Study On Contrastive Search And Contrastive Decoding For Open-ended Text Generation☆26Updated 5 months ago
- TBC☆26Updated 2 years ago
- ☆11Updated last year
- ☆44Updated 2 months ago
- ☆23Updated last year
- ☆21Updated last year
- Github repository for "FELM: Benchmarking Factuality Evaluation of Large Language Models" (NeurIPS 2023)☆54Updated 10 months ago
- [NeurIPS 2022] Generating Training Data with Language Models: Towards Zero-Shot Language Understanding☆62Updated 2 years ago
- [EMNLP 2022] Code for our paper “ZeroGen: Efficient Zero-shot Learning via Dataset Generation”.☆16Updated 2 years ago
- ☆42Updated 10 months ago
- The project page for "SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables"☆19Updated 11 months ago
- ☆27Updated 8 months ago
- A Structured Span Selector (NAACL 2022). A structured span selector with a WCFG for span selection tasks (coreference resolution, semanti…☆21Updated 2 years ago
- The LM Contamination Index is a manually created database of contamination evidences for LMs.☆76Updated 7 months ago
- ☆16Updated last year
- ☆18Updated last year
- ☆58Updated 2 years ago
- IntructIR, a novel benchmark specifically designed to evaluate the instruction following ability in information retrieval models. Our foc…☆28Updated 5 months ago
- ☆31Updated 7 months ago
- ☆17Updated last year