A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.
☆1,045Nov 4, 2026Updated 2 weeks ago
Alternatives and similar repositories for awesome-evals
Users that are interested in awesome-evals are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Maps: curated list of agent observability tools, standards and platforms (OTel GenAI, tracing, evals, guardrails, MCP)☆64Updated this week
- A curated list of memory systems, frameworks, benchmarks, and research for AI agents.☆68Updated this week
- A curated list of materials on AI guardrails☆122Jul 30, 2026Updated 3 months ago
- ⭐ Curated list of AI Agent frameworks: AutoGen, CrewAI, MetaGPT, LangGraph, Swarms. Multi-Agent Systems, Swarm Intelligence, A2A Protocol…☆114Mar 5, 2026Updated 8 months ago
- A curated collection of resources, papers, tools, and best practices for Context Engineering in AI agents and Large Language Models (LLMs…☆229Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Skills that guide AI coding agents to help you build product-specific AI evals.☆1,816Oct 24, 2026Updated 3 weeks ago
- Framework for evaluating and improving agents☆7,205Updated this week
- 🛠️ Awesome tools & guides for harness engineering.☆4,755Nov 9, 2026Updated last week
- SkillsBench evaluates how well skills work and how effective agents are at using them.☆1,926Updated this week
- Research infra for creating RL environments, post-training, and evals.☆368Updated this week
- ☆23Oct 24, 2026Updated 3 weeks ago
- Live Knowledge Base of LLM Agents for Science☆34Nov 1, 2026Updated 2 weeks ago
- Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (i…☆11,804Updated this week
- Skills for AI Evals to compliment the course: AI Evals For Engineers & PMs☆1,676Aug 16, 2026Updated 3 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: arti…☆444Updated this week
- Repository for results and data (coming soon!) for ClawsBench☆39Aug 28, 2026Updated 2 months ago
- Build and deploy AI Agents on Cloudflare☆6,000Updated this week
- 🚀 An open-source, hands-on curriculum bridging the gap from basic RL concepts to LLM alignment, RLVR, and advanced Agentic systems.☆4,926Oct 23, 2026Updated 3 weeks ago
- awesome autoresearch list☆793Updated this week
- Hierarchal Agent Loop Optimizer☆1,217Oct 26, 2026Updated 3 weeks ago
- AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI☆122,514Updated this week
- Plug-and-play PyTorch transformer research framework. Swap attention, FFN, norm, positional, connection, projection, optimizer, scheduler…☆18Oct 25, 2026Updated 3 weeks ago
- A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.☆4,374Oct 17, 2026Updated last month
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Our library for RL environments + evals☆4,793Updated this week
- What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers…☆27,299Sep 21, 2025Updated last year
- Official Compound Engineering plugin for Claude Code, Codex, Cursor, and more☆26,135Updated this week
- Omnigent is an open-source AI agent framework and meta-harness: orchestrate Claude Code, Codex, Cursor, Pi, and custom agents — swap harn…☆11,400Updated this week
- Inspect: A framework for large language model evaluations☆3,277Updated this week
- Harness engineering beginner tutorial, from 0 to 1☆25,414Updated this week
- AI agents running research on single-GPU nanochat training automatically☆100,239Mar 26, 2026Updated 7 months ago
- Track 52,867+ AI benchmark, eval, dataset, and data-quality records from 41 public sources, with linked evidence and daily updates.☆411Updated this week
- ☆24Jul 20, 2026Updated 3 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- The LLM Evaluation Framework☆19,547Updated this week
- Autonomous experiment loop extension for pi☆8,246Oct 7, 2026Updated last month
- AgentENV (AENV) is a distributed platform for running agent environments at scale.☆3,754Updated this week
- A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchma…☆1,243Aug 14, 2026Updated 3 months ago
- Standards for defining and evaluating agent behavior☆379Jul 29, 2026Updated 3 months ago
- A self-improving RLM agent for coding workflows and long-running autonomous tasks.☆22,827Updated this week
- "AgentSpace: Human + Agents. One Team. One Workspace"☆1,034Jul 24, 2026Updated 3 months ago
Are these results useful?