cofe-ai / nanoLM
An Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales
☆10Updated 5 months ago
Related projects ⓘ
Alternatives and complementary repositories for nanoLM
- We introduce ScaleQuest, a scalable, novel and cost-effective data synthesis method to unleash the reasoning capability of LLMs.☆51Updated 3 weeks ago
- Towards Systematic Measurement for Long Text Quality☆29Updated 2 months ago
- Source code of "Reasons to Reject? Aligning Language Models with Judgments"☆56Updated 8 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆73Updated 8 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆61Updated 11 months ago
- ☆17Updated 2 years ago
- GSM-Plus: Data, Code, and Evaluation for Enhancing Robust Mathematical Reasoning in Math Word Problems.☆48Updated 4 months ago
- [ICLR'24 spotlight] Tool-Augmented Reward Modeling☆36Updated 8 months ago
- Code and data for paper "Context-faithful Prompting for Large Language Models".☆39Updated last year
- This repository contains the joint use of CPO and SimPO method for better reference-free preference learning methods.☆35Updated 3 months ago
- [ICML'2024] Can AI Assistants Know What They Don't Know?☆70Updated 9 months ago
- ☆89Updated last month
- Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models [NeurIPS 2024]☆49Updated last week
- Data and code for our paper "Why Does the Effective Context Length of LLMs Fall Short?"☆64Updated last week
- Suri: Multi-constraint instruction following for long-form text generation (EMNLP’24)☆17Updated 2 weeks ago
- A collection of instruction data and scripts for machine translation.☆20Updated last year
- [EMNLP 2024] Source code for the paper "Learning Planning-based Reasoning with Trajectory Collection and Process Rewards Synthesizing".☆43Updated 2 months ago
- ☆69Updated last year
- The official repository of the Omni-MATH benchmark.☆52Updated 3 weeks ago
- L-CITEEVAL: DO LONG-CONTEXT MODELS TRULY LEVERAGE CONTEXT FOR RESPONDING?☆19Updated last month
- 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training☆88Updated last month
- [ACL 2024] Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning☆31Updated 3 months ago
- ☆27Updated 10 months ago
- ☆25Updated last month
- LongHeads: Multi-Head Attention is Secretly a Long Context Processor☆28Updated 7 months ago
- ☆32Updated 8 months ago
- the instructions and demonstrations for building a formal logical reasoning capable GLM☆52Updated 2 months ago
- BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval☆57Updated last month
- [EMNLP 2023] C-STS: Conditional Semantic Textual Similarity☆66Updated 6 months ago
- ☆39Updated 7 months ago