CodeCreator / datatoolsLinks
Common tools for data processing
☆20Updated last month
Alternatives and similar repositories for datatools
Users that are interested in datatools are comparing it to the libraries listed below
Sorting:
- QRHead: Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking☆24Updated last month
- [ICLR'25] Data and code for our paper "Why Does the Effective Context Length of LLMs Fall Short?"☆77Updated 10 months ago
- 🚢 Data Toolkit for Sailor Language Models☆94Updated 7 months ago
- Long Context Extension and Generalization in LLMs☆61Updated last year
- [EMNLP'24] LongHeads: Multi-Head Attention is Secretly a Long Context Processor☆30Updated last year
- Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"☆24Updated 5 months ago
- ☆80Updated 3 weeks ago
- Official repository for ACL 2025 paper "Model Extrapolation Expedites Alignment"☆75Updated 4 months ago
- ☆39Updated 6 months ago
- Official implementation for 'Extending LLMs’ Context Window with 100 Samples'☆80Updated last year
- Pile Deduplication Code☆19Updated 2 years ago
- Suri: Multi-constraint instruction following for long-form text generation (EMNLP’24)☆25Updated 2 weeks ago
- Official repo of dataset-decomposition paper [NeurIPS 2024]☆20Updated 9 months ago
- [NeurIPS 2024] OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI☆105Updated 7 months ago
- Code for Zero-Shot Tokenizer Transfer☆138Updated 9 months ago
- ☆53Updated last year
- Sotopia-RL: Reward Design for Social Intelligence☆43Updated 2 months ago
- Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model☆43Updated 2 weeks ago
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆35Updated last month
- Codebase for Instruction Following without Instruction Tuning☆36Updated last year
- Code for ICML 25 paper "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"☆42Updated 3 months ago
- This is the official repository for Inheritune.☆115Updated 8 months ago
- Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…☆75Updated last year
- Organize the Web: Constructing Domains Enhances Pre-Training Data Curation☆67Updated 5 months ago
- 🫧 Code for Holistic Reasoning with Long-Context LMs: A Benchmark for Database Operations on Massive Textual Data (Maekawa*, Iso* et al.…☆12Updated 7 months ago
- The code for the paper: "Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models"☆54Updated last year
- FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions☆48Updated last year
- ☆12Updated 11 months ago
- Benchmarking Benchmark Leakage in Large Language Models☆55Updated last year
- Official implementation for DenseMixer: Improving MoE Post-Training with Precise Router Gradient☆58Updated 2 months ago