A benchmark for evaluating AI agents on realistic business workflows
☆170Jul 16, 2026Updated 2 weeks ago
Alternatives and similar repositories for AutomationBench
Users that are interested in AutomationBench are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Toolathlon-Gym for testing AI agents real-world tool-use capabilities across diverse MCP servers.☆142Jul 22, 2026Updated last week
- [ICLR 2026] The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution☆449Updated this week
- Official eval scripts for JobBench☆35Jul 18, 2026Updated 2 weeks ago
- A benchmark for general-purpose terminal-use agents.☆45Updated this week
- Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours☆484Jul 22, 2026Updated last week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Measuring and evolving with the frontier of agent work☆436Updated this week
- ☆20Jun 18, 2026Updated last month
- A curated list of awesome Harbor ecosystem projects☆51May 29, 2026Updated 2 months ago
- ☆81Updated this week
- Terminal-Bench Science: Evaluating AI Agents on Complex Real-World Scientific Workflows in the Terminal☆230Updated this week
- ☆43Jun 17, 2026Updated last month
- MCP Atlas☆137Updated this week
- FrontierSmith, a new system that uses AI to synthesize open-ended coding problems at scale☆50May 30, 2026Updated 2 months ago
- Harness for running and evaluating AI agents against RL environments☆227Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Benchmarking Language Agents Under Controllable and Extreme Context Growth☆51Apr 29, 2026Updated 3 months ago
- Codebase for EnterpriseOps-Gym from ServiceNow☆112Jul 5, 2026Updated 3 weeks ago
- SWE-Marathon: an ultra long-horizon SWE benchmark☆125Updated this week
- Approximating the joint distribution of language models via MCTS☆22Nov 3, 2024Updated last year
- τ²-Bench-Verified is a corrected and verified version of the original τ²-bench benchmark. This release addresses issues discovered in the…☆43Apr 2, 2026Updated 4 months ago
- Reproducing R1 for Code with Reliable Rewards☆13Apr 9, 2025Updated last year
- Source code for the TMLR paper "Black-Box Prompt Learning for Pre-trained Language Models"☆59Sep 7, 2023Updated 2 years ago
- LLM RL envs done right, plus some training code☆17Feb 1, 2026Updated 6 months ago
- Code and data used in the paper: "Training on Incorrect Synthetic Data via RL Scales LLM Math Reasoning Eight-Fold"☆32Jun 16, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- The place for RIZZ☆12Mar 10, 2025Updated last year
- ☆29Apr 30, 2024Updated 2 years ago
- FrontierSWE is an ultra long-horizon coding agent benchmark that tests implementation, performance eng and ML research☆202Jul 17, 2026Updated 2 weeks ago
- ☆139Mar 20, 2025Updated last year
- A benchmark for evaluating LLMs on open-ended CS problems. Exploring the Next Frontier of Computer Science.☆288Updated this week
- [EMNLP-2025] R1-Zero on ANY TASK☆32Nov 9, 2025Updated 8 months ago
- EdgeBench: Unveiling scaling laws of learning from real-world environments☆407Jul 17, 2026Updated 2 weeks ago
- Benchmarking multimodal agents on realistic, ultra-challenging visual scenarios requiring long-horizon hybrid tool use.☆70Mar 10, 2026Updated 4 months ago
- [ACL'25 Findings] Official repo for "HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation Task"☆40Apr 7, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Data recipes and robust infrastructure for training AI agents☆273Updated this week
- Realistic examples of building evals and optimizing agents with Harbor☆158Apr 23, 2026Updated 3 months ago
- A lightweight, high-performance AI/RAG workspace and autonomous agent framework implemented in Go. Inspired by the Leann RAG backend arch…☆36Updated this week
- 🍼 Official implementation of Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts☆41Sep 29, 2024Updated last year
- ☆219Dec 19, 2025Updated 7 months ago
- Terminal-Bench 2.1☆63Jul 22, 2026Updated last week
- New testbed of interactive SWE tasks for coding agents, set in a realistic multi-turn developer driven environment☆24Jun 30, 2026Updated last month