Alternatives and similar repositories for super-benchmark

Users that are interested in super-benchmark are comparing it to the libraries listed below

• MLE-Dojo / MLE-Dojo
  ☆53Updated last week
• OSU-NLP-Group / llm-planning-eval
  [ACL'24] Code and data of paper "When is Tree Search Useful for LLM Planning? It Depends on the Discriminator"
  ☆54Updated last year
• neulab / data-agora
  [arXiv preprint] Official Repository for "Evaluating Language Models as Synthetic Data Generators"
  ☆33Updated 6 months ago
• r-three / RAD
  Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model
  ☆43Updated last year
• oriyor / assistantbench
  Implementation of the paper: "AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?"
  ☆57Updated 6 months ago
• GAIR-NLP / scaleeval
  Scalable Meta-Evaluation of LLMs as Evaluators
  ☆42Updated last year
• ContextualAI / CLAIR_and_APO
  Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment
  ☆57Updated 9 months ago
• HazyResearch / aioli
  Aioli: A unified optimization framework for language model data mixing
  ☆27Updated 5 months ago
• allenai / easy-to-hard-generalization
  Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"
  ☆48Updated last year
• martin-wey / CodeUltraFeedback
  CodeUltraFeedback: aligning large language models to coding preferences
  ☆71Updated last year
• OSU-NLP-Group / Middleware
  Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments (EMNLP'2024)
  ☆37Updated 5 months ago
• kaistAI / Janus
  [NeurIPS 2024] Train LLMs with diverse system messages reflecting individualized preferences to generalize to unseen system messages
  ☆48Updated 6 months ago
• XiangLi1999 / AutoBencher
  ☆29Updated 11 months ago
• nathanhu0 / CaMeLS
  Codebase for Context-aware Meta-learned Loss Scaling (CaMeLS). https://arxiv.org/abs/2305.15076.
  ☆25Updated last year
• PrasannS / rlhf-length-biases
  ☆28Updated last year
• SalesforceAIResearch / swecomm
  ☆27Updated 5 months ago
• kaistAI / InstructIR
  IntructIR, a novel benchmark specifically designed to evaluate the instruction following ability in information retrieval models. Our foc…
  ☆32Updated last year
• OSU-NLP-Group / In-Context-Reranking
  [ICLR'25] "Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers"
  ☆22Updated 2 months ago
• ScalerLab / JudgeBench
  ☆85Updated 7 months ago
• amazon-science / llm-code-preference
  Training and Benchmarking LLMs for Code Preference.
  ☆33Updated 7 months ago
• test-time-interaction / TTI
  ☆36Updated 2 weeks ago
• Yu-Fangxu / FoR
  [ICML 2025] Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examples
  ☆95Updated 2 weeks ago
• allenai / MacGyver
  Code and Data for the NAACL 24 paper: MacGyver: Are Large Language Models Creative Problem Solvers?
  ☆28Updated last year
• scandukuri / assistant-gate
  ☆25Updated last year
• skywalker023 / fantom
  👻 Code and benchmark for our EMNLP 2023 paper - "FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions"
  ☆55Updated last year
• austrian-code-wizard / c3po
  ☆27Updated this week
• Nanami18 / Snowballed_Hallucination
  ☆44Updated 9 months ago
• hamishivi / automated-instruction-selection
  Exploration of automated dataset selection approaches at large scales.
  ☆45Updated 3 months ago
• zorazrw / odex
  [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation
  ☆48Updated last year
• Zayne-sprague / MuSR
  ☆44Updated 10 months ago