philschmid / ai-agent-benchmark-compendiumLinks
Compendium of over 50 benchmarks for evaluating AI agents, categorized into Function Calling & Tool Use, General Assistant & Reasoning, Coding & Software Engineering, and Computer Interaction.
☆88Updated 3 months ago
Alternatives and similar repositories for ai-agent-benchmark-compendium
Users that are interested in ai-agent-benchmark-compendium are comparing it to the libraries listed below
Sorting:
- Leveraging DSPy for AI-driven task understanding and solution generation, the Self-Discover Framework automates problem-solving through r…☆73Updated 3 months ago
- Doing simple retrieval from LLM models at various context lengths to measure accuracy☆108Updated 4 months ago
- A user interface for DSPy☆211Updated 4 months ago
- A DSPy-based implementation of the tree of thoughts method (Yao et al., 2023) for generating persuasive arguments☆98Updated 4 months ago
- Simple Graph Memory for AI applications☆90Updated 8 months ago
- Official homepage for "Self-Harmonized Chain of Thought" (NAACL 2025)☆92Updated last year
- Context Engineering Course with DSPy☆214Updated 6 months ago
- ☆37Updated 9 months ago
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆115Updated 10 months ago
- Writing Blog Posts with Generative Feedback Loops!☆50Updated last year
- Testing speed and accuracy of RAG with, and without Cross Encoder Reranker.☆50Updated 2 years ago
- Dynamic Metadata based RAG Framework☆78Updated 2 months ago
- A collection of Compound Retrieval Systems implemented with DSPy and Weaviate.☆94Updated this week
- A fast, local, and secure approach for training LLMs for coding tasks using GRPO with WebAssembly and interpreter feedback.☆41Updated 10 months ago
- Simple examples using Argilla tools to build AI☆57Updated last year
- ☆80Updated last year
- ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)☆90Updated 2 months ago
- ☆57Updated 2 years ago
- ☆39Updated last year
- A mcp server that uses the Osmosis-Apply-1.7B model to apply code merges☆53Updated 7 months ago
- ☆51Updated 5 months ago
- Very minimal (and stateless) agent framework☆44Updated last year
- ☆74Updated last year
- Official Repo for CRMArena and CRMArena-Pro☆132Updated last week
- LLM reads a paper and produce a working prototype☆60Updated 10 months ago
- ☆43Updated 3 months ago
- Sample applications built on the Graphlit Platform☆76Updated 4 months ago
- ☆11Updated 11 months ago
- Python library to use Pleias-RAG models☆68Updated 9 months ago
- Easily convert agents and orchestrators from existing agent frameworks to A2A servers☆58Updated 8 months ago