allenai / olmesLinks
Reproducible, flexible LLM evaluations
☆248Updated 2 months ago
Alternatives and similar repositories for olmes
Users that are interested in olmes are comparing it to the libraries listed below
Sorting:
- Benchmarking LLMs with Challenging Tasks from Real Users☆241Updated 10 months ago
- A simple unified framework for evaluating LLMs☆245Updated 5 months ago
- LOFT: A 1 Million+ Token Long-Context Benchmark☆211Updated 3 months ago
- 🌾 OAT: A research-friendly framework for LLM online alignment, including reinforcement learning, preference learning, etc.☆461Updated this week
- Automatic evals for LLMs☆526Updated 2 months ago
- The HELMET Benchmark☆172Updated last month
- ☆190Updated 5 months ago
- RewardBench: the first evaluation tool for reward models.☆634Updated 3 months ago
- Code and example data for the paper: Rule Based Rewards for Language Model Safety☆196Updated last year
- PyTorch building blocks for the OLMo ecosystem☆292Updated this week
- The official evaluation suite and dynamic data release for MixEval.☆245Updated 10 months ago
- Official repository for ACL 2025 paper "ProcessBench: Identifying Process Errors in Mathematical Reasoning"☆171Updated 4 months ago
- ☆315Updated 3 months ago
- Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"☆226Updated last week
- open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality☆210Updated last year
- BABILong is a benchmark for LLM evaluation using the needle-in-a-haystack approach.☆212Updated 2 weeks ago
- [ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning☆362Updated last year
- ☆98Updated 10 months ago
- Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate" [COLM 2025]☆171Updated 2 months ago
- Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".☆216Updated last month
- Code for NeurIPS'24 paper 'Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization'☆229Updated 2 months ago
- Implementation of paper Data Engineering for Scaling Language Models to 128K Context☆473Updated last year
- A project to improve skills of large language models☆560Updated this week
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆310Updated last year
- DSIR large-scale data selection framework for language model training☆258Updated last year
- Official code for "MAmmoTH2: Scaling Instructions from the Web" [NeurIPS 2024]☆148Updated 10 months ago
- ☆163Updated last year
- [ICLR 2025] BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval☆167Updated last week
- Code for STaR: Bootstrapping Reasoning With Reasoning (NeurIPS 2022)☆209Updated 2 years ago
- Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.☆431Updated last year