ibm-ecosystem-engineering / JudgeIt-LLM-as-a-Judge
Automation Framework using LLM-as-a-judge to Scale Eval of Gen AI solutions (RAG, Multi-turn, Query Rewrite, Text2SQL etc.); that is a good proxy for human judgement.
☆23Updated last month
Alternatives and similar repositories for JudgeIt-LLM-as-a-Judge:
Users that are interested in JudgeIt-LLM-as-a-Judge are comparing it to the libraries listed below
- Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute…☆48Updated 7 months ago
- ☆45Updated 4 months ago
- 🔎 A deep-dive into HyDE for Advanced LLM RAG + 💡 Introducing AutoHyDE, a semi-supervised framework to improve the effectiveness, covera…☆32Updated 10 months ago
- LLM reads a paper and produce a working prototype☆48Updated 2 weeks ago
- ☆76Updated 8 months ago
- Code and data for "StructLM: Towards Building Generalist Models for Structured Knowledge Grounding" (COLM 2024)☆76Updated 4 months ago
- Using open source LLMs to build synthetic datasets for direct preference optimization☆57Updated 11 months ago
- Mixing Language Models with Self-Verification and Meta-Verification☆101Updated 2 months ago
- Codebase accompanying the Summary of a Haystack paper.☆74Updated 5 months ago
- ☆48Updated 3 months ago
- Writing Blog Posts with Generative Feedback Loops!☆47Updated 11 months ago
- A RAG that can scale 🧑🏻💻☆11Updated 8 months ago
- Versatile framework designed to streamline the integration of your models, as well as those sourced from Hugging Face, into complex progr…☆28Updated 2 months ago
- ☆24Updated last month
- ☆141Updated 7 months ago
- ☆24Updated last year
- EMNLP 2024 "Re-reading improves reasoning in large language models". Simply repeating the question to get bidirectional understanding for…☆24Updated 2 months ago
- High level library for batched embeddings generation, blazingly-fast web-based RAG and quantized indexes processing ⚡☆64Updated 3 months ago
- ☆32Updated 7 months ago
- ARAGOG- Advanced RAG Output Grading. Exploring and comparing various Retrieval-Augmented Generation (RAG) techniques on AI research paper…☆102Updated 10 months ago
- ☆22Updated 2 weeks ago
- WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting.☆37Updated 6 months ago
- [SIGIR 2024 (Demo)] CoSearchAgent: A Lightweight Collborative Search Agent with Large Language Models☆22Updated last year
- PyTorch implementation for MRL☆18Updated last year
- 🔧 Compare how Agent systems perform on several benchmarks. 📊🚀☆75Updated 4 months ago
- ☆18Updated 4 months ago
- Preference-based Recursive Language Modeling for Exploratory Optimization of Reasoning☆50Updated last month
- Universal text classifier for generative models☆22Updated 6 months ago
- ☆73Updated last month
- Measuring RAG solutions throughput and latency☆15Updated 6 months ago