IBM / eval-assistLinks
EvalAssist is an open-source project that simplifies using large language models as evaluators (LLM-as-a-Judge) of the output of other large language models by supporting users in iteratively refining evaluation criteria in a web-based user experience.
☆91Updated last week
Alternatives and similar repositories for eval-assist
Users that are interested in eval-assist are comparing it to the libraries listed below
Sorting:
- Low latency, High Accuracy, Custom Query routers for Humans and Agents. Built by Prithivi Da☆117Updated 7 months ago
- 🦄 Unitxt is a Python library for enterprise-grade evaluation of AI performance, offering the world's largest catalog of tools and data …☆211Updated last week
- LangFair is a Python library for conducting use-case level LLM bias and fairness assessments☆240Updated 2 weeks ago
- Granite Snack Cookbook -- easily consumable recipes (python notebooks) that showcase the capabilities of the Granite models☆302Updated this week
- A framework for fine-tuning retrieval-augmented generation (RAG) systems.☆133Updated last week
- This repository stems from our paper, “Cataloguing LLM Evaluations”, and serves as a living, collaborative catalogue of LLM evaluation fr…☆18Updated last year
- Efficiently find the best-suited language model (LM) for your NLP task☆127Updated 3 months ago
- ☆146Updated last year
- A comprehensive guide to LLM evaluation methods designed to assist in identifying the most suitable evaluation techniques for various use…☆147Updated 3 weeks ago
- Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute…☆50Updated last year
- RAGElo is a set of tools that helps you selecting the best RAG-based LLM agents by using an Elo ranker☆121Updated last week
- A blueprint for AI development, focusing on applied examples of RAG, information extraction, analysis and fine-tuning in the age of LLMs …☆61Updated 9 months ago
- SynthGenAI - Package for Generating Synthetic Datasets using LLMs.☆50Updated this week
- Simple UI for debugging correlations of text embeddings☆298Updated 5 months ago
- Collection of resources for RL and Reasoning☆26Updated 9 months ago
- A small library of LLM judges☆299Updated 3 months ago
- ☆96Updated 7 months ago
- Generalist and Lightweight Model for Text Classification☆164Updated 4 months ago
- ☆80Updated last year
- A reasoning assistant for your STEM education☆23Updated 7 months ago
- Enterprise-grade memory framework for LLMs featuring GPU-optimized inference, vector storage, and automated scaling. Enables hyper-person…☆88Updated 6 months ago
- CUGA is an open-source generalist agent for the enterprise, supporting complex task execution on web and APIs, OpenAPI/MCP integrations, …☆173Updated this week
- ☆44Updated last month
- 🧠🔗 From idea to production in just few lines: Graph-Based Programmable Neuro-Symbolic LM Framework - a production-first LM framework bu…☆352Updated this week
- Benchmark various LLM Structured Output frameworks: Instructor, Mirascope, Langchain, LlamaIndex, Fructose, Marvin, Outlines, etc on task…☆179Updated last year
- Synthetic Text Dataset Generation for LLM projects☆43Updated last week
- all code examples in the blog posts☆21Updated 9 months ago
- ☆12Updated last year
- Repository for the paper "MALADE: Orchestration of LLM-powered Agents with Retrieval Augmented Generation for Pharmacovigilance"☆19Updated 8 months ago
- A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings.☆170Updated last week