Alternatives and similar repositories for LLM-as-a-Judge

Users that are interested in LLM-as-a-Judge are comparing it to the libraries listed below

• ParticleMedia / RAGTruth
  Github repository for "RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models"
  ☆197Updated 9 months ago
• weizhepei / InstructRAG
  [ICLR 2025] InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales
  ☆120Updated 7 months ago
• ScalerLab / JudgeBench
  ☆98Updated 10 months ago
• ZitongYang / Synthetic_Continued_Pretraining
  Code implementation of synthetic continued pretraining
  ☆127Updated 8 months ago
• tianyang-x / SaySelf
  Public code repo for paper "SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales"
  ☆110Updated 11 months ago
• tianyi-lab / Reflection_Tuning
  [ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
  ☆362Updated last year
• TIGER-AI-Lab / CritiqueFineTuning
  Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate" [COLM 2025]
  ☆171Updated 2 months ago
• rxlqn / awesome-llm-self-reflection
  augmented LLM with self reflection
  ☆131Updated last year
• facebookresearch / ReasonIR
  Official repository for paper "ReasonIR Training Retrievers for Reasoning Tasks".
  ☆198Updated 2 months ago
• shizhediao / R-Tuning
  [NAACL 2024 Outstanding Paper] Source code for the NAACL 2024 paper entitled "R-Tuning: Instructing Large Language Models to Say 'I Don't…
  ☆116Updated last year
• kyegomez / Lets-Verify-Step-by-Step
  "Improving Mathematical Reasoning with Process Supervision" by OPENAI
  ☆113Updated last week
• allenai / WildBench
  Benchmarking LLMs with Challenging Tasks from Real Users
  ☆241Updated 10 months ago
• google-deepmind / loft
  LOFT: A 1 Million+ Token Long-Context Benchmark
  ☆209Updated 3 months ago
• matthewrenze / self-reflection
  Self-Reflection in LLM Agents: Effects on Problem-Solving Performance
  ☆83Updated 9 months ago
• zorazrw / awesome-tool-llm
  ☆239Updated last year
• TIGER-AI-Lab / MAmmoTH2
  Official code for "MAmmoTH2: Scaling Instructions from the Web" [NeurIPS 2024]
  ☆147Updated 10 months ago
• Hannibal046 / xRAG
  [Neurips2024] Source code for xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
  ☆152Updated last year
• xlang-ai / BRIGHT
  [ICLR 2025] BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval
  ☆164Updated 3 months ago
• MozerWang / Loong
  [EMNLP 2024 (Oral)] Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
  ☆139Updated 10 months ago
• SqueezeAILab / LLM2LLM
  [ACL 2024] LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement
  ☆189Updated last year
• NVlabs / Tool-N1
  ☆205Updated 3 months ago
• Open-Source-O1 / o1_Reasoning_Patterns_Study
  ☆103Updated 9 months ago
• SalesforceAIResearch / FaithEval
  ☆48Updated 3 months ago
• SALT-NLP / collaborative-gym
  Framework and toolkits for building and evaluating collaborative agents that can work together with humans.
  ☆97Updated 5 months ago
• xiaoya-li / Instruction-Tuning-Survey
  Project for the paper entitled `Instruction Tuning for Large Language Models: A Survey`
  ☆185Updated last month
• knoveleng / open-rs
  Official repo for paper: "Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't"
  ☆257Updated 4 months ago
• ulab-uiuc / MARBLE
  (ACL 2025 Main) Code for MultiAgentBench : Evaluating the Collaboration and Competition of LLM agents https://www.arxiv.org/pdf/2503.019…
  ☆157Updated this week
• GAIR-NLP / scaleeval
  Scalable Meta-Evaluation of LLMs as Evaluators
  ☆42Updated last year
• sail-sg / CPO
  [NeurIPS 2024] The official implementation of paper: Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs.
  ☆129Updated 5 months ago
• kevinwu23 / StanfordClashEval
  ☆37Updated 7 months ago