p-lambda / dsir
DSIR large-scale data selection framework for language model training
☆242Updated 9 months ago
Alternatives and similar repositories for dsir:
Users that are interested in dsir are comparing it to the libraries listed below
- Pytorch implementation of DoReMi, a method for optimizing the data mixture weights in language modeling datasets☆313Updated last year
- ☆251Updated last year
- [EMNLP 2023] The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning☆221Updated last year
- Official repository of NEFTune: Noisy Embeddings Improves Instruction Finetuning☆388Updated 8 months ago
- [EMNLP 2023] Adapting Language Models to Compress Long Contexts☆292Updated 4 months ago
- A Survey on Data Selection for Language Models☆201Updated 3 months ago
- All available datasets for Instruction Tuning of Large Language Models☆240Updated last year
- Scaling Data-Constrained Language Models☆330Updated 3 months ago
- Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 100K Tokens": https://arxiv.org/abs/2402.13718☆303Updated 3 months ago
- [ICML'24] Data and code for our paper "Training-Free Long-Context Scaling of Large Language Models"☆377Updated 3 months ago
- RewardBench: the first evaluation tool for reward models.☆491Updated last week
- Codes and Data for Scaling Relationship on Learning Mathematical Reasoning with Large Language Models☆233Updated 4 months ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆154Updated 7 months ago
- A large-scale, fine-grained, diverse preference dataset (and models).☆325Updated last year
- [ACL'24 Outstanding] Data and code for L-Eval, a comprehensive long context language models evaluation benchmark☆368Updated 6 months ago
- Code and data for "Lost in the Middle: How Language Models Use Long Contexts"☆324Updated last year
- Implementation of paper Data Engineering for Scaling Language Models to 128K Context☆447Updated 9 months ago
- Deita: Data-Efficient Instruction Tuning for Alignment [ICLR2024]☆524Updated last month
- Code and Data for "Long-context LLMs Struggle with Long In-context Learning"☆97Updated 6 months ago
- ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models☆171Updated 3 months ago
- What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets☆200Updated 2 months ago
- [ICML 2024] Selecting High-Quality Data for Training Language Models☆154Updated 6 months ago
- Official PyTorch implementation of DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024)☆171Updated 3 months ago
- ☆268Updated last year
- Multipack distributed sampler for fast padding-free training of LLMs☆184Updated 5 months ago
- Official code for "MAmmoTH2: Scaling Instructions from the Web" [NeurIPS 2024]☆129Updated 2 months ago
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆123Updated last year
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆204Updated 7 months ago
- Self-Alignment with Principle-Following Reward Models☆150Updated 10 months ago
- [ICML 2024] LESS: Selecting Influential Data for Targeted Instruction Tuning☆400Updated 2 months ago