PAIR-code / llm-comparator
LLM Comparator is an interactive data visualization tool for evaluating and analyzing LLM responses side-by-side, developed by the PAIR team.
☆415Updated 2 months ago
Alternatives and similar repositories for llm-comparator:
Users that are interested in llm-comparator are comparing it to the libraries listed below
- Evaluate your LLM's response with Prometheus and GPT4 💯☆911Updated last month
- awesome synthetic (text) datasets☆272Updated 5 months ago
- Automatically evaluate your LLMs in Google Colab☆615Updated 11 months ago
- This project showcases an LLMOps pipeline that fine-tunes a small-size LLM model to prepare for the outage of the service LLM.☆303Updated 3 weeks ago
- SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models☆509Updated 9 months ago
- Automatic evals for LLMs☆373Updated this week
- Automated Evaluation of RAG Systems☆579Updated 3 weeks ago
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆1,438Updated last week
- Model, Code & Data for the EMNLP'23 paper "Making Large Language Models Better Data Creators"☆131Updated last year
- ☆515Updated 5 months ago
- A Lightweight Library for AI Observability☆241Updated 2 months ago
- A comprehensive guide to LLM evaluation methods designed to assist in identifying the most suitable evaluation techniques for various use…☆114Updated last week
- TapeAgents is a framework that facilitates all stages of the LLM Agent development lifecycle☆257Updated this week
- Banishing LLM Hallucinations Requires Rethinking Generalization☆273Updated 9 months ago
- Github repository for "RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models"☆170Updated 4 months ago
- Code and Data for Tau-Bench☆437Updated 3 months ago
- 🤗 Benchmark Large Language Models Reliably On Your Data☆240Updated last week
- Let's build better datasets, together!☆259Updated 4 months ago
- Tutorial for building LLM router☆193Updated 9 months ago
- ☆162Updated 4 months ago
- Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard a…☆1,143Updated 3 months ago
- Generative Representational Instruction Tuning☆620Updated last month
- A reading list on LLM based Synthetic Data Generation 🔥☆1,246Updated 2 months ago
- An agent benchmark with tasks in a simulated software company.☆294Updated 2 weeks ago
- Benchmark various LLM Structured Output frameworks: Instructor, Mirascope, Langchain, LlamaIndex, Fructose, Marvin, Outlines, etc on task…☆163Updated 7 months ago
- DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models. 🤖💤☆1,010Updated 2 months ago
- Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.☆197Updated 2 weeks ago
- [ICLR 2025] Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing. Your efficient and high-quality synthetic data …☆681Updated last month
- Initiative to evaluate and rank the most popular LLMs across common task types based on their propensity to hallucinate.☆107Updated 7 months ago
- Official repository for ORPO☆448Updated 10 months ago