Benchmarking Open-Ended Inference Optimization by AI Agents
☆44Jul 6, 2026Updated 2 months ago
Alternatives and similar repositories for InferenceBench
Users that are interested in InferenceBench are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injections☆22Jul 2, 2026Updated 2 months ago
- ☆13Jun 18, 2024Updated 2 years ago
- Python package for serving a local search engine. One command to download and serve a datastore---that's it 😎.☆26Jun 6, 2025Updated last year
- ☆51Updated this week
- Estimate MFU for DeepSeekV3☆26Jan 5, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A Top-Down Profiler for GPU Applications☆24Feb 29, 2024Updated 2 years ago
- Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours☆554Updated this week
- ☆23Jun 18, 2026Updated 2 months ago
- A Difficulty-Calibrated Benchmark for Building Terminal Agents☆30Feb 20, 2026Updated 6 months ago
- Training terminal-agents☆299Updated this week
- Simple template for quick prototyping and standardization of deep learning projects☆11Dec 29, 2023Updated 2 years ago
- ☆14Aug 9, 2023Updated 3 years ago
- ☆49Oct 1, 2024Updated last year
- ☆56Jun 25, 2026Updated 2 months ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Official eval scripts for JobBench☆52Updated this week
- Measuring and evolving with the frontier of agent work☆673Updated this week
- A suite of interpretability tasks to evaluate agents using Scribe for notebook access☆18Oct 2, 2025Updated 11 months ago
- An evaluation suite for Retrieval-Augmented Generation (RAG).☆25Apr 26, 2025Updated last year
- Code for safety test in "Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates"☆22Sep 21, 2025Updated 11 months ago
- A research workbench for developing and testing attacks against large language models, with a focus on prompt injection vulnerabilities a…☆60Jul 24, 2026Updated last month
- A collection of judges for evaluating LLM model output for safety & toxicity with a standardized API.☆16Jan 7, 2026Updated 8 months ago
- Official implementation of Our NeurIPS 2024 Paper "Boundary Matters: A Bi-Level Active Finetuning Method"☆14Feb 11, 2025Updated last year
- ☆115Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Code for the paper "Harnessing Discrete Representations for Continual Reinforcement Learning"☆16Jun 16, 2024Updated 2 years ago
- context-efficient terminal agent powered by an RLM☆61Feb 7, 2026Updated 7 months ago
- Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks☆96Aug 29, 2026Updated 2 weeks ago
- ACL24☆11Jun 7, 2024Updated 2 years ago
- Code for the paper "Distinguishing the Knowable from the Unknowable with Language Models"☆12Jul 18, 2026Updated last month
- ☆16Dec 9, 2023Updated 2 years ago
- Comparison of gradient estimation techniques for black-box adversarial examples☆11Oct 31, 2018Updated 7 years ago
- Code companion for the RL Post-Training Handbook - training reasoning models on a single GPU☆20Jan 30, 2026Updated 7 months ago
- ☆19Apr 28, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Code and data for NAACL 2025 paper "IHEval: Evaluating Language Models on Following the Instruction Hierarchy"☆18Feb 25, 2025Updated last year
- The Automated LLM Speedrunning Benchmark measures how well LLM agents can reproduce previous innovations and discover new ones in languag…☆145May 6, 2026Updated 4 months ago
- ☆12Sep 16, 2024Updated last year
- Provable Worst Case Guarantees for the Detection of Out-of-Distribution Data☆13Sep 20, 2022Updated 3 years ago
- FrontierSmith, a new system that uses AI to synthesize open-ended coding problems at scale☆52May 30, 2026Updated 3 months ago
- An agent harness that compiles a model into one provably-correct, self-retargeting CUDA megakernel and self-tunes it past cuBLAS at batch…☆138Jun 29, 2026Updated 2 months ago
- JMLR Cover Letter Template☆10Dec 15, 2021Updated 4 years ago