A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.
☆1,017Oct 27, 2026Updated this week
Alternatives and similar repositories for awesome-evals
Users that are interested in awesome-evals are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Maps: curated list of agent observability tools, standards and platforms (OTel GenAI, tracing, evals, guardrails, MCP)☆62Updated this week
- A curated list of memory systems, frameworks, benchmarks, and research for AI agents.☆65Updated this week
- A curated list of materials on AI guardrails☆120Jul 30, 2026Updated 2 months ago
- ⭐ Curated list of AI Agent frameworks: AutoGen, CrewAI, MetaGPT, LangGraph, Swarms. Multi-Agent Systems, Swarm Intelligence, A2A Protocol…☆109Mar 5, 2026Updated 7 months ago
- A curated collection of resources, papers, tools, and best practices for Context Engineering in AI agents and Large Language Models (LLMs…☆223Updated this week
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Live Knowledge Base of LLM Agents for Science☆34Oct 22, 2026Updated last week
- Framework for evaluating and improving agents☆6,619Updated this week
- Skills that guide AI coding agents to help you build product-specific AI evals.☆1,575Updated this week
- ☆21Updated this week
- SkillsBench evaluates how well skills work and how effective agents are at using them.☆1,877Updated this week
- Research infra for creating RL environments, post-training, and evals.☆357Updated this week
- Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: arti…☆389Updated this week
- Skills for AI Evals to compliment the course: AI Evals For Engineers & PMs☆1,675Aug 16, 2026Updated 2 months ago
- 🛠️ Awesome tools & guides for harness engineering.☆4,506Aug 19, 2026Updated 2 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (i…☆11,664Updated this week
- Repository for results and data (coming soon!) for ClawsBench☆37Aug 28, 2026Updated 2 months ago
- 🚀 An open-source, hands-on curriculum bridging the gap from basic RL concepts to LLM alignment, RLVR, and advanced Agentic systems.☆4,817Oct 23, 2026Updated last week
- AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI☆118,754Updated this week
- Hierarchal Agent Loop Optimizer☆1,202Updated this week
- Track 33,707+ AI benchmark, eval, dataset, and data-quality records from 40 public sources, with linked evidence and daily updates.☆309Updated this week
- awesome autoresearch list☆790Updated this week
- Harness engineering beginner tutorial, from 0 to 1☆22,358Oct 22, 2026Updated last week
- Standards for defining and evaluating agent behavior☆363Jul 29, 2026Updated 3 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Plug-and-play PyTorch transformer research framework. Swap attention, FFN, norm, positional, connection, projection, optimizer, scheduler…☆18Updated this week
- A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.☆4,296Oct 17, 2026Updated last week
- ☆39Oct 6, 2026Updated 3 weeks ago
- AgentENV (AENV) is a distributed platform for running agent environments at scale.☆3,705Updated this week
- copy markdown sections from pi responses☆13Jul 13, 2026Updated 3 months ago
- Official Compound Engineering plugin for Claude Code, Codex, Cursor, and more☆25,771Updated this week
- Omnigent is an open-source AI agent framework and meta-harness: orchestrate Claude Code, Codex, Cursor, Pi, and custom agents — swap harn…☆11,114Updated this week
- A plain-language writing skill for AI agents, with a revision view that shows what changed.☆729Aug 23, 2026Updated 2 months ago
- My Toy Language☆33Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers…☆26,916Sep 21, 2025Updated last year
- A self-improving RLM agent for coding workflows and long-running autonomous tasks.☆22,420Updated this week
- Autonomous experiment loop extension for pi☆8,203Oct 7, 2026Updated 3 weeks ago
- "AgentSpace: Human + Agents. One Team. One Workspace"☆1,021Jul 24, 2026Updated 3 months ago
- Build and deploy AI Agents on Cloudflare☆5,916Updated this week
- ☆24Jul 20, 2026Updated 3 months ago
- AI agents running research on single-GPU nanochat training automatically☆98,818Mar 26, 2026Updated 7 months ago