A benchmark for evaluating AI agents on realistic business workflows
☆279Aug 4, 2026Updated last month
Alternatives and similar repositories for AutomationBench
Users that are interested in AutomationBench are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Toolathlon-Gym for testing AI agents real-world tool-use capabilities across diverse MCP servers.☆147Jul 22, 2026Updated last month
- [ICLR 2026] The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution☆482Aug 18, 2026Updated 3 weeks ago
- Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours☆554Updated this week
- Official eval scripts for JobBench☆52Updated this week
- A benchmark for general-purpose terminal-use agents.☆47Aug 7, 2026Updated last month
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains☆571Updated this week
- Measuring and evolving with the frontier of agent work☆673Updated this week
- MCP Atlas☆153Updated this week
- ☆23Jun 18, 2026Updated 2 months ago
- ☆115Updated this week
- ☆45Jun 17, 2026Updated 2 months ago
- FrontierSmith, a new system that uses AI to synthesize open-ended coding problems at scale☆52May 30, 2026Updated 3 months ago
- Harness for running and evaluating AI agents against RL environments☆270Updated this week
- MCPMark is a comprehensive, stress-testing MCP benchmark designed to evaluate model and agent capabilities in real-world MCP use.☆460Jun 12, 2026Updated 3 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆19Mar 25, 2026Updated 5 months ago
- New testbed of interactive SWE tasks for coding agents, set in a realistic multi-turn developer driven environment☆26Jun 30, 2026Updated 2 months ago
- Pushing Test-Time Scaling Limits of Deep Search with Asymmetric Verification☆22Oct 8, 2025Updated 11 months ago
- Benchmarking multimodal agents on realistic, ultra-challenging visual scenarios requiring long-horizon hybrid tool use.☆74Mar 10, 2026Updated 6 months ago
- Codebase for EnterpriseOps-Gym from ServiceNow☆126Aug 20, 2026Updated 3 weeks ago
- Approximating the joint distribution of language models via MCTS☆22Nov 3, 2024Updated last year
- SWE-Marathon: an ultra long-horizon SWE benchmark☆152Updated this week
- Source code for the TMLR paper "Black-Box Prompt Learning for Pre-trained Language Models"☆59Sep 7, 2023Updated 3 years ago
- τ²-Bench-Verified is a corrected and verified version of the original τ²-bench benchmark. This release addresses issues discovered in the…☆52Apr 2, 2026Updated 5 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Code and data used in the paper: "Training on Incorrect Synthetic Data via RL Scales LLM Math Reasoning Eight-Fold"☆32Jun 16, 2024Updated 2 years ago
- EdgeBench: Unveiling scaling laws of learning from real-world environments☆443Updated this week
- The place for RIZZ☆12Mar 10, 2025Updated last year
- ☆29Apr 30, 2024Updated 2 years ago
- Environments by the Prime Intellect Research Team☆128Updated this week
- FrontierSWE is an ultra long-horizon coding agent benchmark that tests implementation, performance eng and ML research☆229Aug 13, 2026Updated last month
- ☆139Mar 20, 2025Updated last year
- A benchmark for evaluating LLMs on open-ended CS problems. Exploring the Next Frontier of Computer Science.☆313Updated this week
- [EMNLP-2025] R1-Zero on ANY TASK☆32Nov 9, 2025Updated 10 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- This is AlpaGasus2-QLoRA based on LLaMA2 with AlpaGasus mechanism using QLoRA!☆15Nov 22, 2023Updated 2 years ago
- [ACL'25 Findings] Official repo for "HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation Task"☆40Apr 7, 2025Updated last year
- Data recipes and robust infrastructure for training AI agents☆290Sep 2, 2026Updated last week
- 🍼 Official implementation of Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts☆40Sep 29, 2024Updated last year
- ☆220Dec 19, 2025Updated 8 months ago
- Realistic examples of building evals and optimizing agents with Harbor☆206Apr 23, 2026Updated 4 months ago
- Programmable chat templates for LLM training and inference.☆156Updated this week