A benchmark for evaluating AI agents on realistic business workflows
☆320Aug 4, 2026Updated last month
Alternatives and similar repositories for AutomationBench
Users that are interested in AutomationBench are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Toolathlon-Gym for testing AI agents real-world tool-use capabilities across diverse MCP servers.☆151Jul 22, 2026Updated 2 months ago
- [ICLR 2026] The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution☆491Aug 18, 2026Updated last month
- Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours☆580Updated this week
- A curated list of awesome Harbor ecosystem projects☆54May 29, 2026Updated 4 months ago
- A benchmark for general-purpose terminal-use agents.☆48Updated this week
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains☆666Updated this week
- Official eval scripts for JobBench☆56Sep 14, 2026Updated 2 weeks ago
- Measuring and evolving with the frontier of agent work☆834Updated this week
- MCP Atlas☆157Updated this week
- ☆23Jun 18, 2026Updated 3 months ago
- ☆120Updated this week
- ☆46Jun 17, 2026Updated 3 months ago
- FrontierSmith, a new system that uses AI to synthesize open-ended coding problems at scale☆80Updated this week
- Harness for running and evaluating AI agents against RL environments☆282Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- MCPMark is a comprehensive, stress-testing MCP benchmark designed to evaluate model and agent capabilities in real-world MCP use.☆461Jun 12, 2026Updated 3 months ago
- ☆20Mar 25, 2026Updated 6 months ago
- New testbed of interactive SWE tasks for coding agents, set in a realistic multi-turn developer driven environment☆28Jun 30, 2026Updated 3 months ago
- Codebase for EnterpriseOps-Gym from ServiceNow☆133Aug 20, 2026Updated last month
- Benchmarking multimodal agents on realistic, ultra-challenging visual scenarios requiring long-horizon hybrid tool use.☆74Mar 10, 2026Updated 6 months ago
- Approximating the joint distribution of language models via MCTS☆22Nov 3, 2024Updated last year
- SWE-Marathon: an ultra long-horizon SWE benchmark☆164Sep 25, 2026Updated last week
- τ²-Bench-Verified is a corrected and verified version of the original τ²-bench benchmark. This release addresses issues discovered in the…☆54Apr 2, 2026Updated 6 months ago
- LLM RL envs done right, plus some training code☆17Feb 1, 2026Updated 8 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- EdgeBench: Unveiling scaling laws of learning from real-world environments☆457Sep 20, 2026Updated last week
- The place for RIZZ☆12Mar 10, 2025Updated last year
- ☆29Apr 30, 2024Updated 2 years ago
- Environments by the Prime Intellect Research Team☆129Updated this week
- FrontierSWE is an ultra long-horizon coding agent benchmark that tests implementation, performance eng and ML research☆231Aug 13, 2026Updated last month
- ☆139Mar 20, 2025Updated last year
- A benchmark for evaluating LLMs on open-ended CS problems. Exploring the Next Frontier of Computer Science.☆322Updated this week
- [EMNLP-2025] R1-Zero on ANY TASK☆33Nov 9, 2025Updated 10 months ago
- This is AlpaGasus2-QLoRA based on LLaMA2 with AlpaGasus mechanism using QLoRA!☆15Nov 22, 2023Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- [ACL'25 Findings] Official repo for "HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation Task"☆40Apr 7, 2025Updated last year
- Data recipes and robust infrastructure for training AI agents☆297Updated this week
- A lightweight, high-performance AI/RAG workspace and autonomous agent framework implemented in Go. Inspired by the Leann RAG backend arch…☆38Sep 25, 2026Updated last week
- ☆220Dec 19, 2025Updated 9 months ago
- 🍼 Official implementation of Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts☆40Sep 29, 2024Updated 2 years ago
- Realistic examples of building evals and optimizing agents with Harbor☆227Apr 23, 2026Updated 5 months ago
- Programmable chat templates for LLM training and inference.☆159Updated this week