A simple plug-in framework that corrects bias and computes confidence intervals in reporting LLM-as-a-judge evaluation, and an adaptive algorithm that efficiently allocates calibration samples to reduce uncertainty in estimates.
☆76Nov 27, 2025Updated 4 months ago
Alternatives and similar repositories for LLM-judge-reporting
Users that are interested in LLM-judge-reporting are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A vanilla implementation of ReAct: Synergizing Reasoning and Acting in Language Models☆17Mar 26, 2025Updated last year
- ☆14Dec 12, 2024Updated last year
- MV-RAG combines retrieval with multi-view generation to create accurate 3D-consistent visuals. By retrieving reference images and text, i…☆24Nov 29, 2025Updated 4 months ago
- Wyckoff Transformer: Generation of Symmetric Crystals [ICML 2025]☆29Apr 8, 2026Updated last week
- Providing the answer to "How to do patching on all available SAEs on GPT-2?". It is an official repository of the implementation of the p…☆13Jan 26, 2025Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Synthetic data generation and benchmark implementation for "Episodic Memories Generation and Evaluation Benchmark for Large Language Mode…☆66Oct 3, 2025Updated 6 months ago
- Bayesian Optimization with multiple tiers of objectives, which can flexibly depend on model inputs and outputs☆14Sep 10, 2025Updated 7 months ago
- KernelBench v2: Can LLMs Write GPU Kernels? - Benchmark with Torch -> Triton (and more!) problems☆23Jul 4, 2025Updated 9 months ago
- Python powered music controlling webpage with websockets and bottle py (works with spotify, vlc, audacious, and others)☆11Jun 9, 2017Updated 8 years ago
- Coordinator for stateful sharded services☆190Updated this week
- ☆25Mar 31, 2026Updated 2 weeks ago
- CrysText: A Generative AI Approach for Text-Conditioned Crystal Structure Generation using LLM☆17Nov 3, 2025Updated 5 months ago
- a decentralized dataset generator and manipulator.☆19Updated this week
- Minimalistic implementation for LLM-based chat assistants with Tool Use (function calling) and MCP☆28Jan 19, 2026Updated 2 months ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Official Code for the NeurIPS 2024 paper "FactorSim: Generative Simulation via Factorized Representation"☆14Sep 26, 2024Updated last year
- AtomGPT.org API Usage Examples Paper: https://arxiv.org/abs/2512.11935 Docs: https://atomgptlab.github.io/agapi/☆32Mar 30, 2026Updated 2 weeks ago
- Complex Reasoning with ReAct and LangChain☆12Apr 24, 2024Updated last year
- A collection of plugin for j4status☆14Oct 2, 2023Updated 2 years ago
- Reasoning-based Evaluation and Ranking of Translations.☆20Jul 18, 2025Updated 8 months ago
- ☆12Oct 7, 2020Updated 5 years ago
- [NeurIPS 2024] The official implementation of "Image Copy Detection for Diffusion Models"☆18Oct 1, 2024Updated last year
- ☆16May 31, 2024Updated last year
- An asynchronous streaming data management module for efficient post-training.☆49Updated this week
- Deploy open-source AI quickly and easily - Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- TypeScript SDK for agent portability, discovery and trust based on ERC-8004.☆65Mar 16, 2026Updated last month
- ☆20Jul 23, 2025Updated 8 months ago
- Fast, gpu-accelerated distance transforms☆14Mar 7, 2025Updated last year
- ReCAP: Recursive Context-Aware Reasoning and Planning for Large Language Model Agents, NeurIPS 2025☆35Nov 15, 2025Updated 5 months ago
- Applying SAEs for fine-grained control☆26Dec 15, 2024Updated last year
- Unofficial entropix impl for Gemma2 and Llama and Qwen2 and Mistral☆17Jan 12, 2025Updated last year
- Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication☆21Mar 21, 2024Updated 2 years ago
- ☆28Aug 27, 2025Updated 7 months ago
- ☆14Mar 8, 2025Updated last year
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- The implementation of "RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question" [ACL 2023]☆16Apr 7, 2024Updated 2 years ago
- Touch camera for Bevy that supports drag and pinch to zoom☆13Feb 21, 2024Updated 2 years ago
- ☆15Apr 17, 2025Updated 11 months ago
- AgentBudget is the ulimit for AI agents. Just like Unix systems have ulimit to prevent a single process from consuming all system resourc…☆98Apr 5, 2026Updated last week
- ☆13Aug 5, 2025Updated 8 months ago
- Make things clickable☆40Apr 6, 2016Updated 10 years ago
- Embedding language models in probability space via log-likelihood vectors☆16Oct 25, 2025Updated 5 months ago