A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.
☆829Jul 31, 2026Updated 2 weeks ago
Alternatives and similar repositories for awesome-evals
Users that are interested in awesome-evals are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆18Aug 9, 2026Updated last week
- A plain-language writing skill for AI agents, with a revision view that shows what changed.☆333Updated this week
- Skills for building LLM evals. Starts with interactive error discovery: build a review app, sample diverse traces, and organize human ann…☆220Updated this week
- 🚀 An open-source, hands-on curriculum bridging the gap from basic RL concepts to LLM alignment, RLVR, and advanced Agentic systems.☆3,971Updated this week
- ☆22Jul 20, 2026Updated 3 weeks ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- [ICML 2026] NITP: Next Implicit Token Prediction for LLM Pre-training☆35May 26, 2026Updated 2 months ago
- "AgentSpace: Human + Agents. One Team. One Workspace"☆933Jul 24, 2026Updated 3 weeks ago
- awesome autoresearch list☆685Updated this week
- Skills for AI Evals to compliment the course: AI Evals For Engineers & PMs☆1,665Updated this week
- AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI☆91,467Updated this week
- ChatGPT for Mac, living in your menubar.☆631Jun 26, 2026Updated last month
- A review-first reference architecture for AI-assisted personal content knowledge systems☆116Jul 11, 2026Updated last month
- Autonomous experiment loop extension for pi☆7,609Jul 15, 2026Updated last month
- Hierarchal Agent Loop Optimizer☆1,143Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆19Aug 23, 2025Updated 11 months ago
- A Python port of Pi’s minimalist coding agent.☆2,340Updated this week
- Premium hand-drawn animated architecture diagram Codex skill☆890Jun 26, 2026Updated last month
- Reusable AI video production skills library for creation, recreation, motion design, openers, and QA.☆610Jul 14, 2026Updated last month
- Omnigent is an open-source AI agent framework and meta-harness: orchestrate Claude Code, Codex, Cursor, Pi, and custom agents — swap harn…☆8,913Updated this week
- Official Compound Engineering plugin for Claude Code, Codex, Cursor, and more☆24,306Updated this week
- Head Vis Public Release☆41May 4, 2026Updated 3 months ago
- Framework for evaluating and improving agents☆4,282Updated this week
- AI agents running research on single-GPU nanochat training automatically☆93,937Mar 26, 2026Updated 4 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- turns your codebase into an autoresearch loop — discovers what to measure, instruments the benchmark, then runs tree search with parallel…☆1,372Jul 17, 2026Updated 3 weeks ago
- What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers…☆25,331Sep 21, 2025Updated 10 months ago
- Plan and autonomously build a software task end-to-end. One ready-to-paste /goal, adaptive phase count, memory preload + writeback, 3-str…☆661Jun 8, 2026Updated 2 months ago
- slime is an LLM post-training framework for RL Scaling.☆8,056Updated this week
- Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard a…☆2,142Dec 3, 2025Updated 8 months ago
- Our library for RL environments + evals☆4,521Updated this week
- "QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks"☆244Aug 5, 2026Updated last week
- SkillsBench evaluates how well skills work and how effective agents are at using them.☆1,685Jul 23, 2026Updated 3 weeks ago
- An Agent Skill for designing cross-platform desktop apps that feel native — distilled from Raycast's 2.0 deep-dive and reverse engineerin…☆1,876May 30, 2026Updated 2 months ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Symphony turns project work into isolated, autonomous implementation runs, allowing teams to manage work instead of supervising coding ag…☆26,693Updated this week
- Design visual, review-gated agent loops for Claude Code before you run them.☆704Aug 9, 2026Updated last week
- Composable computational-science methodology skills for AI research agents — pre-registration over TDD. A science-domain reimplementation…☆289Updated this week
- Research infra for creating RL environments, post-training, and evals.☆324Updated this week
- Agent Reinforcement Trainer: train multi-step agents for real-world tasks using GRPO. Give your agents on-the-job training. Reinforcement…☆10,593Updated this week
- The Little Book of Reinforcement Learning☆1,514Jul 14, 2026Updated last month
- Field Theory CLI for bookmarks, Library, commands, and agent workflows☆1,977Jul 16, 2026Updated last month