A small library of LLM judges
☆340Jul 31, 2025Updated last year
Alternatives and similar repositories for judges
Users that are interested in judges are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Inference-time scaling for LLMs-as-a-judge.☆353Nov 5, 2025Updated last year
- Synthetic data curation for post-training and structured data extraction☆1,763Updated this week
- ☆12Apr 26, 2024Updated 2 years ago
- ☆583May 21, 2026Updated 5 months ago
- Ranger helps you see the forest among the trees - Ranger is an effect-size meta analysis library creating beautiful forest plots!☆12Jun 12, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- FastFit ⚡ When LLMs are Unfit Use FastFit ⚡ Fast and Effective Text Classification with Many Classes☆222Oct 10, 2026Updated 3 weeks ago
- TaskWeaver Plugins☆12Jan 28, 2024Updated 2 years ago
- Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard a…☆2,165Dec 3, 2025Updated 11 months ago
- Fast Multimodal Semantic Deduplication & Filtering☆978Oct 4, 2026Updated last month
- Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets☆5,193Updated this week
- Automatic evals for LLMs☆615Feb 24, 2026Updated 8 months ago
- a Python library that uses Reinforcement Learning (RL) to train LLMs.☆44Jul 12, 2026Updated 3 months ago
- ☆28May 15, 2024Updated 2 years ago
- Gorilla: An API store for LLMs☆10Jun 28, 2024Updated 2 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Distill thinking dataset more compactly and accurately!☆39Jun 6, 2025Updated last year
- [ICLR 2025 Spotlight] An open-sourced LLM judge for evaluating LLM-generated answers.☆445Feb 11, 2025Updated last year
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆3,417Updated this week
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆36Apr 17, 2025Updated last year
- Web UI & Backend for Data Annotations in Aya☆31Mar 16, 2024Updated 2 years ago
- Skill to annotate and create ai judges from agent logs☆18Oct 28, 2025Updated last year
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.☆1,643Dec 20, 2025Updated 10 months ago
- never forget anything again! combine AI and intelligent tooling for a local knowledge base to track catalogue, annotate, and plan for you…☆36May 14, 2024Updated 2 years ago
- Loom TUI and Character Lab for developing AI characters from base models: branch text, curate anthologies, simulate conversations, and ev…☆13Oct 4, 2026Updated last month
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- [ICML 2025] Teaching Language Models to Critique via Reinforcement Learning☆125May 6, 2025Updated last year
- ☆25Jun 5, 2025Updated last year
- ☆16Oct 17, 2024Updated 2 years ago
- DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models. 🤖💤☆1,126Feb 2, 2025Updated last year
- structured outputs for llms☆14,099Updated this week
- ☆161Dec 2, 2024Updated last year
- ☆82Jun 5, 2024Updated 2 years ago
- Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreement☆15Oct 8, 2025Updated last year
- Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-…☆3,974May 17, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks [ICML 2025]☆12Jun 11, 2025Updated last year
- Our library for RL environments + evals☆4,774Updated this week
- Late Interaction Models Training & Retrieval☆904Jul 23, 2026Updated 3 months ago
- StAtutory Reasoning Assessment☆17Dec 8, 2022Updated 3 years ago
- A blueprint for AI development, focusing on applied examples of RAG, information extraction, analysis and fine-tuning in the age of LLMs …☆67Feb 6, 2025Updated last year
- ☆23Dec 24, 2025Updated 10 months ago
- Streamline on-policy/off-policy distillation workflows in a few lines of code☆112Aug 5, 2026Updated 3 months ago