A benchmark for evaluating AI agents on realistic business workflows
☆128Jul 7, 2026Updated last week
Alternatives and similar repositories for AutomationBench
Users that are interested in AutomationBench are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A benchmark for general-purpose terminal-use agents.☆43Jun 30, 2026Updated 2 weeks ago
- Official eval scripts for JobBench☆26Jul 5, 2026Updated last week
- Measuring agents' ability to get work done on a computer☆308Updated this week
- Toolathlon-Gym for testing AI agents real-world tool-use capabilities across diverse MCP servers.☆137Apr 2, 2026Updated 3 months ago
- ☆66Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆18Jun 18, 2026Updated 3 weeks ago
- MCP Atlas☆116Updated this week
- ☆43Jun 17, 2026Updated 3 weeks ago
- FrontierSmith, a new system that uses AI to synthesize open-ended coding problems at scale☆45May 30, 2026Updated last month
- Resources and paper list for 'Scaling Environments for Agents'. This repository accompanies our survey on how environments contribute to …☆71Jan 28, 2026Updated 5 months ago
- Codebase for EnterpriseOps-Gym from ServiceNow☆108Jul 5, 2026Updated last week
- SWE-Marathon: an ultra long-horizon SWE benchmark☆103Updated this week
- Convert GitHub PRs into Harbor tasks☆69Updated this week
- Reproducing R1 for Code with Reliable Rewards☆13Apr 9, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Source code for the TMLR paper "Black-Box Prompt Learning for Pre-trained Language Models"☆59Sep 7, 2023Updated 2 years ago
- Approximating the joint distribution of language models via MCTS☆22Nov 3, 2024Updated last year
- Code and data used in the paper: "Training on Incorrect Synthetic Data via RL Scales LLM Math Reasoning Eight-Fold"☆32Jun 16, 2024Updated 2 years ago
- FrontierSWE is an ultra long-horizon coding agent benchmark that tests implementation, performance eng and ML research☆174Updated this week
- The place for RIZZ☆12Mar 10, 2025Updated last year
- [AAAI26]: DS SERVE: The Largest Open Vector Store over Pretain Data; A Framework for Efficient and Scalable Neural Retrieval☆53Jan 28, 2026Updated 5 months ago
- [ICLR 2026] The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution☆427Updated this week
- ☆138Mar 20, 2025Updated last year
- Benchmarking multimodal agents on realistic, ultra-challenging visual scenarios requiring long-horizon hybrid tool use.☆65Mar 10, 2026Updated 4 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- This is AlpaGasus2-QLoRA based on LLaMA2 with AlpaGasus mechanism using QLoRA!☆15Nov 22, 2023Updated 2 years ago
- Data recipes and robust infrastructure for training AI agents☆259Updated this week
- 🍼 Official implementation of Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts☆41Sep 29, 2024Updated last year
- Autocomp: Optimize any AI kernel, anywhere.☆144Updated this week
- Realistic examples of building evals and optimizing agents with Harbor☆134Apr 23, 2026Updated 2 months ago
- A JAX-accelerated implementation of the Procedural Content Generation via Reinforcement Learning (PCGRL) framework. We train RL agents to…☆15Nov 26, 2025Updated 7 months ago
- [ACL'25 Findings] Official repo for "HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation Task"☆40Apr 7, 2025Updated last year
- Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering☆63Dec 5, 2024Updated last year
- Terminal-Bench 2.1☆40Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Code for the ACL-2022 paper "Knowledge Neurons in Pretrained Transformers"☆177May 4, 2024Updated 2 years ago
- Ludic – an LLM-RL library for the era of experience☆66Jan 9, 2026Updated 6 months ago
- Source code for the paper "Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data"☆20Feb 24, 2024Updated 2 years ago
- Harness for running and evaluating AI agents against RL environments☆216Updated this week
- [ICLR 2025] "Training LMs on Synthetic Edit Sequences Improves Code Synthesis" (Piterbarg, Pinto, Fergus)☆19Feb 11, 2025Updated last year
- Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering☆51Aug 2, 2022Updated 3 years ago
- a version of baby agi using dspy and typed predictors☆15Mar 9, 2024Updated 2 years ago