Alternatives and similar repositories for MMLU-Pro:

Users that are interested in MMLU-Pro are comparing it to the libraries listed below

• ZubinGou / math-evaluation-harness
  A simple toolkit for benchmarking LLMs on mathematical reasoning tasks. 🧮✨
  ☆191Updated 11 months ago
• tianyi-lab / Reflection_Tuning
  [ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
  ☆349Updated 6 months ago
• allenai / WildBench
  Benchmarking LLMs with Challenging Tasks from Real Users
  ☆219Updated 4 months ago
• TIGER-AI-Lab / CritiqueFineTuning
  Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate"
  ☆131Updated last month
• allenai / olmes
  Reproducible, flexible LLM evaluations
  ☆180Updated this week
• THUDM / LongAlign
  [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs
  ☆247Updated 3 months ago
• JinjieNi / MixEval
  The official evaluation suite and dynamic data release for MixEval.
  ☆233Updated 4 months ago
• huggingface / cosmopedia
  ☆502Updated 4 months ago
• kaistAI / CoT-Collection
  [EMNLP 2023] The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning
  ☆236Updated last year
• OpenBMB / InfiniteBench
  Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 100K Tokens": https://arxiv.org/abs/2402.13718
  ☆313Updated 6 months ago
• Re-Align / URIAL
  ☆307Updated 9 months ago
• HKUNLP / ChunkLlama
  [ICML'24] Data and code for our paper "Training-Free Long-Context Scaling of Large Language Models"
  ☆393Updated 5 months ago
• huggingface / Math-Verify
  ☆559Updated last week
• swj0419 / detect-pretrain-code
  This repository provides an original implementation of Detecting Pretraining Data from Large Language Models by *Weijia Shi, *Anirudh Aji…
  ☆218Updated last year
• voidism / DoLa
  Official implementation for the paper "DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models"
  ☆475Updated 2 months ago
• wuhy68 / Parameter-Efficient-MoE
  Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
  ☆141Updated 6 months ago
• QwenLM / AutoIF
  ☆264Updated 8 months ago
• fanqiwan / FuseAI
  FuseAI Project
  ☆550Updated 2 months ago
• openai / safety-rbr-code-and-data
  Code and example data for the paper: Rule Based Rewards for Language Model Safety
  ☆183Updated 8 months ago
• sail-sg / oat
  🌾 OAT: A research-friendly framework for LLM online alignment, including preference learning, reinforcement learning, etc.
  ☆283Updated this week
• allenai / reward-bench
  RewardBench: the first evaluation tool for reward models.
  ☆532Updated last month
• cmu-l3 / l1
  L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
  ☆148Updated last week
• GAIR-NLP / ProX
  Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"
  ☆229Updated last month
• andyrdt / refusal_direction
  Code and results accompanying the paper "Refusal in Language Models Is Mediated by a Single Direction".
  ☆195Updated 5 months ago
• RUC-GSAI / YuLan-Mini
  A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.
  ☆165Updated last week
• THUDM / T1
  Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  ☆95Updated 2 months ago
• sail-sg / oat-zero
  A lightweight reproduction of DeepSeek-R1-Zero with indepth analysis of self-reflection behavior.
  ☆212Updated this week
• dwzhu-pku / LongEmbed
  LongEmbed: Extending Embedding Models for Long Context Retrieval (EMNLP 2024)
  ☆131Updated 4 months ago
• OpenBMB / Eurus
  ☆312Updated 6 months ago
• eddycmu / demystify-long-cot
  ☆260Updated last week