Python package for estimating a CIs for metrics evaluated by LLM-as-Judges.
☆103May 25, 2025Updated last year
Alternatives and similar repositories for judgy
Users that are interested in judgy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆338Mar 24, 2026Updated 7 months ago
- An intro to Braintrust and homework implementations using Braintrust as the backend☆34Mar 25, 2026Updated 7 months ago
- ☆72Aug 5, 2025Updated last year
- Skills for AIs to do evals, error analysis, and more.☆12Feb 19, 2026Updated 9 months ago
- Writing-style rules mined from 63 style guides, organized, deduped, and browsable☆16Jun 19, 2026Updated 4 months ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- ☆17Dec 16, 2025Updated 11 months ago
- Skills for AI Evals to compliment the course: AI Evals For Engineers & PMs☆1,676Aug 16, 2026Updated 3 months ago
- Skills that guide AI coding agents to help you build product-specific AI evals.☆1,816Oct 24, 2026Updated 3 weeks ago
- ☆39Aug 4, 2025Updated last year
- ☆60Jan 28, 2026Updated 9 months ago
- DEPRECATED--all functionality moved to nbdev☆14Aug 3, 2022Updated 4 years ago
- Specflow is a structured but flexible process methodology for building with software agents☆157Feb 10, 2026Updated 9 months ago
- Example of using the SAINT architecture in fastai☆12Aug 25, 2021Updated 5 years ago
- Distributed multi-agent framework for event-driven, graph-based computation. Elixir/Python, NATS event streaming, modular operator/XCS ar…☆14Mar 25, 2026Updated 7 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- DSPy module for OpenAI Codex SDK - signature-driven agentic workflows☆166Dec 8, 2025Updated 11 months ago
- Low-Cost LLM-Powered Data Processing with Theoretical Guarantees☆61Jun 11, 2026Updated 5 months ago
- A collection of Compound Retrieval Systems implemented with DSPy and Weaviate.☆100Jun 1, 2026Updated 5 months ago
- A naive attempt at generating better prompts for opencode using DSPy☆19Nov 29, 2025Updated 11 months ago
- Homework assignments for Evaluating and Improving AI Agents☆98Oct 10, 2026Updated last month
- Pharmaceutical decision science is the practical bridge between commercial data and commercial decisions. It focuses on commercial decisi…☆13Jul 23, 2026Updated 3 months ago
- ☆12Mar 10, 2022Updated 4 years ago
- Open-source system for the design and analysis of controlled experiments, intended for use inside Claude Code.☆12Aug 27, 2026Updated 2 months ago
- Comprehensive LLM evaluation framework: GPQA Diamond to Chatbot Arena. Tests all major models equally, easily extensible.☆18Aug 22, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Quick tutorial showing how to fine-tune Llama3.1 with nothing but free tools and text data. All code included in ipynb. For a step by ste…☆10Aug 17, 2024Updated 2 years ago
- Collection of useful Dagger modules☆36Updated this week
- The missing quality toolkit for Agent Skills☆15Jul 21, 2026Updated 3 months ago
- Hosted Amazon Seller & Vendor MCP server. Connect Claude, ChatGPT, Cursor, Codex, Gemini, GitHub Copilot to Amazon SP-API and Ads API dat…☆18Updated this week
- Showcase of a line-of-business agent with evaluation framework☆14Oct 24, 2025Updated last year
- SynthGenAI - Package for Generating Synthetic Datasets using LLMs.☆59Nov 24, 2025Updated 11 months ago
- Minimal agent runtime built with DSPy modules and a thin Python loop. Includes CLI, FastAPI server, and eval harness with OpenAI/Ollama s…☆78Apr 25, 2026Updated 6 months ago
- Dashboard for tracking Claude Code API usage with USD/JPY conversion☆20Jun 5, 2025Updated last year
- Repository for experiments with MetaProd2Vec and related algorithms.☆59Mar 16, 2019Updated 7 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- AI Product Analyst Plus — full system + premium skills for bootcamp and consulting☆21Jul 15, 2026Updated 4 months ago
- Code for a workshop on geo-visualization at UC Berkeley's iSchool on 2017-04-10☆16Apr 10, 2017Updated 9 years ago
- A Terraform module for scheduling Vertex Pipeline runs using Google Cloud Scheduler☆13Dec 7, 2022Updated 3 years ago
- A system for agentic LLM-powered data processing and ETL☆4,259Updated this week
- 📸 Browser-based tool for culling and ranking large photo libraries. Remove blurry shots 🗑️, collapse burst duplicates 📚, and surface y…☆31Sep 13, 2026Updated 2 months ago
- A DSPy Adapter for exact-fidelity prompt templates with full control over messages.☆54Feb 23, 2026Updated 8 months ago
- Integrating SSE with NVIDIA Triton Inference Server using a Python backend and Zephyr model. There is very less documentation how to use …☆10May 29, 2024Updated 2 years ago
Are these results useful?