Alternatives and similar repositories for agentic-benchmarks

Users that are interested in agentic-benchmarks are comparing it to the libraries listed below

• HyperPotatoNeo / RSA
  ☆81Updated 3 months ago
• oriyor / assistantbench
  Implementation of the paper: "AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?"
  ☆66Updated last year
• bespokelabsai / verifiers
  Verifiers for LLM Reinforcement Learning
  ☆80Updated 9 months ago
• sunnweiwei / PPP-Agent
  ☆92Updated 2 months ago
• data-for-agents / insta
  Official Repo for InSTA: Towards Internet-Scale Training For Agents
  ☆55Updated 6 months ago
• austrian-code-wizard / c3po
  ☆29Updated 3 weeks ago
• kilian-group / phantom-wiki
  Python package for generating datasets to evaluate reasoning and retrieval of large language models
  ☆18Updated 3 months ago
• SalesforceAIResearch / CodeTree
  Code for the paper: CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models
  ☆30Updated 9 months ago
• arcee-ai / DAM
  ☆55Updated last year
• ZeroSumEval / ZeroSumEval
  A framework for pitting LLMs against each other in an evolving library of games ⚔
  ☆34Updated 9 months ago
• salesforce / summary-of-a-haystack
  Codebase accompanying the Summary of a Haystack paper.
  ☆80Updated last year
• allenai / infinigram-api
  ☆92Updated last month
• para-lost / ReBase
  ReBase: Training Task Experts through Retrieval Based Distillation
  ☆29Updated 11 months ago
• ContextualAI / CLAIR_and_APO
  Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment
  ☆61Updated last year
• sher222 / LeReT
  Learning to Retrieve by Trying - Source code for Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrieval
  ☆51Updated last year
• AlexCuadron / ThinkingAgent
  Systematic evaluation framework that automatically rates overthinking behavior in large language models.
  ☆95Updated 8 months ago
• safety-research / open-source-alignment-faking
  Open Source Replication of Anthropic's Alignment Faking Paper
  ☆54Updated 9 months ago
• JHU-CLSP / RATIONALYST
  Code for RATIONALYST: Pre-training Process-Supervision for Improving Reasoning https://arxiv.org/pdf/2410.01044
  ☆35Updated last year
• InternLM / SWE-Fixer
  ☆131Updated 8 months ago
• ahstat / episodic-memory-benchmark
  Synthetic data generation and benchmark implementation for "Episodic Memories Generation and Evaluation Benchmark for Large Language Mode…
  ☆63Updated 3 months ago
• allenai / IFBench
  ☆97Updated 2 weeks ago
• ScalingIntelligence / codemonkeys
  ☆59Updated 11 months ago
• shreyashankar / spade-experiments
  Experiments to assess SPADE on different LLM pipelines.
  ☆17Updated last year
• minosvasilias / simple_grpo
  Simple GRPO scripts and configurations.
  ☆59Updated 11 months ago
• AnswerDotAI / ModernBERT-Instruct-mini-cookbook
  ☆53Updated 11 months ago
• lilakk / BLEUBERI
  Official repository for "BLEUBERI: BLEU is a surprisingly effective reward for instruction following"
  ☆31Updated 7 months ago
• benediktstroebl / agent-evals
  ☆25Updated 7 months ago
• kiddyboots216 / lottery-ticket-adaptation
  Lottery Ticket Adaptation
  ☆39Updated last year
• rohinmanvi / Capability-Aware-and-Mid-Generation-Self-Evaluations
  ☆21Updated 5 months ago
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆59Updated 3 months ago