A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.
☆756Jul 1, 2026Updated 3 weeks ago
Alternatives and similar repositories for awesome-evals
Users that are interested in awesome-evals are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆16Jul 11, 2026Updated 2 weeks ago
- A plain-language writing skill for AI agents, with a revision view that shows what changed.☆270Jul 17, 2026Updated last week
- Interactive error analysis skill for AI agents. Studies LLM trace datasets, builds a review UI, monitors annotations, categorizes failure…☆156Jun 25, 2026Updated last month
- 🚀 An open-source, hands-on curriculum bridging the gap from basic RL concepts to LLM alignment, RLVR, and advanced Agentic systems.☆3,349Jul 3, 2026Updated 3 weeks ago
- ☆22Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- [ICML 2026] NITP: Next Implicit Token Prediction for LLM Pre-training☆34May 26, 2026Updated 2 months ago
- "AgentSpace: Human + Agents. One Team. One Workspace"☆846Updated this week
- awesome autoresearch list☆657Updated this week
- Skills for AI Evals to compliment the course: AI Evals For Engineers & PMs☆1,537Jun 10, 2026Updated last month
- AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI☆77,542Updated this week
- ChatGPT for Mac, living in your menubar.☆410Jun 26, 2026Updated last month
- A review-first reference architecture for AI-assisted personal content knowledge systems☆113Jul 11, 2026Updated 2 weeks ago
- Autonomous experiment loop extension for pi☆7,258Jul 15, 2026Updated last week
- Hierarchal Agent Loop Optimizer☆1,116Updated this week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆19Aug 23, 2025Updated 11 months ago
- A Python port of Pi’s minimalist coding agent.☆2,009Updated this week
- Hinbox is a flexible, domain-configurable entity extraction system designed for historians and researchers.☆16Apr 3, 2026Updated 3 months ago
- Premium hand-drawn animated architecture diagram Codex skill☆809Jun 26, 2026Updated last month
- Reusable AI video production skills library for creation, recreation, motion design, openers, and QA.☆571Jul 14, 2026Updated last week
- Omnigent is an open-source AI agent framework and meta-harness: orchestrate Claude Code, Codex, Cursor, Pi, and custom agents — swap harn…☆7,735Updated this week
- Official Compound Engineering plugin for Claude Code, Codex, Cursor, and more☆23,469Updated this week
- Framework for evaluating and improving agents☆3,504Updated this week
- turns your codebase into an autoresearch loop — discovers what to measure, instruments the benchmark, then runs tree search with parallel…☆1,350Jul 17, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- AI agents running research on single-GPU nanochat training automatically☆92,020Mar 26, 2026Updated 4 months ago
- What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers…☆24,762Sep 21, 2025Updated 10 months ago
- Plan and autonomously build a software task end-to-end. One ready-to-paste /goal, adaptive phase count, memory preload + writeback, 3-str…☆653Jun 8, 2026Updated last month
- slime is an LLM post-training framework for RL Scaling.☆7,629Updated this week
- Code repository for ICLR 2026 paper "ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents" (https://ww…☆28Feb 10, 2026Updated 5 months ago
- Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard a…☆2,128Dec 3, 2025Updated 7 months ago
- Our library for RL environments + evals☆4,400Updated this week
- "QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks"☆236Updated this week
- SkillsBench evaluates how well skills work and how effective agents are at using them.☆1,577Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- An Agent Skill for designing cross-platform desktop apps that feel native — distilled from Raycast's 2.0 deep-dive and reverse engineerin…☆1,861May 30, 2026Updated last month
- Symphony turns project work into isolated, autonomous implementation runs, allowing teams to manage work instead of supervising coding ag…☆26,219Updated this week
- Design visual, review-gated agent loops for Claude Code before you run them.☆691Jul 7, 2026Updated 2 weeks ago
- Composable computational-science methodology skills for AI research agents — pre-registration over TDD. A science-domain reimplementation…☆270Updated this week
- Research infra for creating RL environments, post-training, and evals☆302Updated this week
- Agent Reinforcement Trainer: train multi-step agents for real-world tasks using GRPO. Give your agents on-the-job training. Reinforcement…☆10,529Updated this week
- The Little Book of Reinforcement Learning☆1,277Jul 14, 2026Updated last week