CI for LLM agents — change the prompt, swap the model, edit a tool, refactor the loop: replay the eval suite and let statistics decide if anything got worse. OTel tracing, calibrated LLM-as-judge,adversarial simulated users, and a GitHub Actions gate that blocks significant regressions.
☆28Oct 21, 2026Updated last week
Alternatives and similar repositories for agent-eval
Users that are interested in agent-eval are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An AI agent command-line interface — multi-model LLM access, streaming terminal UI, tool calling, permission control, and session persist…☆111Updated this week
- FactCircuit: an auditable claim-and-evidence verification loop with immutable provenance, staged checks, bounded retrieval, and reproduci…☆895Oct 9, 2026Updated 2 weeks ago
- MCPHub Desktop 是一款基于 Tauri 2 构建的桌面客户端，用于在个人电脑上统一管理 MCP服务器、Skills、RAG 知识库。所有进程、配置、密钥与向量数据都运 行/保存在用户自己的机器上，不依赖任何远端服务。☆1,018Oct 15, 2026Updated 2 weeks ago
- An open-source team messenger for groups of up to 30, with spaces, threads, files and voice notes. Built with Next.js and PostgreSQL, dep…☆94Updated this week
- Run Linux-style commands on Windows via deterministic bash→PowerShell translation. No VM, no WSL. MCP server + CLI built for AI agents (C…☆903Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 般若波罗蜜。回到每一个争吵和沉默的节点，看看另一个版本的结局。☆828Sep 17, 2026Updated last month
- 一条命令，把口播毛片变成可二次精修的剪辑工程。☆967Updated this week
- Obsidian AI reading companion - pet chat, themed PPT/HTML export, illustration, research☆49Oct 14, 2026Updated 2 weeks ago
- Agent Skill 步进治理体系：由程序保证 Agent 不跳步骤（step-gate Rust 门控 + 交付门 + 链治理）☆661Sep 5, 2026Updated last month
- SiftGate☆290Oct 22, 2026Updated last week
- Local-first AI agent workspace for multi-agent collaboration, agent orchestration, scoped permissions, evidence-aware runs, and human-in-…☆1,128Updated this week
- ☆98Sep 23, 2026Updated last month
- 一个会总结你习惯的私人 AI agent☆353Sep 17, 2026Updated last month
- CraftStation Agent Runtime Composition System☆859Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- 5–10x faster browser operations: Jev clicks, Codex thinks and verifies. Built at EZCollegeApp.☆1,205Oct 19, 2026Updated last week
- 软件实习项目作业☆990Sep 2, 2026Updated last month
- 大模型驱动的npc决策，npc和玩家平权☆1,187Mar 29, 2026Updated 7 months ago
- 同行 · Together — 面向自主经营、任务与全成就推进的 Stardew Valley AI Agent。原生工具、持续计划、游戏记忆与玩家辅助。☆192Sep 23, 2026Updated last month
- See through the fog of local development ports.☆446Aug 14, 2026Updated 2 months ago
- A lightweight multi-agent system☆435Oct 18, 2026Updated last week
- Official PyTorch implementation of VisionHOPE: Visual Backbones as Self-Modifying Learning Systems.☆1,283Oct 12, 2026Updated 2 weeks ago
- Skill training with a corruption and reconstruction loop: train an agent's skill library, not its weights.☆508Updated this week
- Distill your knowledge, memories, and decisions into an open-source, inspectable AI Agent Twin.☆602Updated this week
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Minimal Claude Code statusline HUD powered by Rust.☆998Aug 13, 2026Updated 2 months ago
- A comprehensive image metadata indexer and viewer designed specifically for AI-generated images. Organize, search, and manage your AI-gen…☆98Updated this week
- A curated directory of translation tools and workflows across travel, web reading, formatted documents, academic research, open-source lo…☆77Oct 20, 2026Updated last week
- ☆741Oct 14, 2026Updated 2 weeks ago
- 希儿最初的家~☆734Updated this week
- Free strategy scorecard for trading backtests. Detect overfitting, weak edge, drawdown risk, and random-timing luck from returns, equity …☆931Updated this week
- MC CityGen: Turn your own Minecraft builds into an entire city.☆476Updated this week
- ☆815Updated this week
- ☆1,342Updated this week
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- OntiCards 是面向企业的 AI 数据中枢（AI Data Hub），围绕业务数据库提供一站式能力：数据源连接、元数据理解、自然语言查询、跨源融合、数据质检、查询审计与数据资产管理等☆590Updated this week
- 基于 Jingyun Studio + DeepSeek Harness (DSH) 打造的一站式 AI 商业化桌面客户端，一个将 AI 智能体 / 技能 / 工作流转化为可交易商品的完整商业化平台客户端。 井云为 DSH 注入了完整的商业闭环：登录注册 → 会员体系 →…☆925Sep 30, 2026Updated 3 weeks ago
- A multi-agent AUTO Research loop for idea discovery, experiment execution, evidence-grounded paper writing, quality audit, and live Web o…☆534Sep 24, 2026Updated last month
- 纯 Java、离线、零依赖的 TXT 小说目录识别算法☆29Jul 17, 2026Updated 3 months ago
- Deep Skill Finder – Real Tests Real Reviews find skills that Real Work☆568Oct 12, 2026Updated 2 weeks ago
- Governance standard and reference toolset for LLM-maintained knowledge corpora☆641Sep 12, 2026Updated last month
- ☆719Oct 8, 2026Updated 3 weeks ago