Alternatives and similar repositories for inspect_ai:

Users that are interested in inspect_ai are comparing it to the libraries listed below

• carlini / yet-another-applied-llm-benchmark
  A benchmark to evaluate language models on questions I've previously asked them to solve.
  ☆1,005Updated 2 months ago
• UKGovernmentBEIS / inspect_evals
  Collection of evals for Inspect AI
  ☆115Updated this week
• prometheus-eval / prometheus-eval
  Evaluate your LLM's response with Prometheus and GPT4 💯
  ☆911Updated last month
• vgel / repeng
  A library for making RepE control vectors
  ☆579Updated 3 months ago
• huggingface / lighteval
  Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
  ☆1,438Updated this week
• huggingface / datatrove
  Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.
  ☆2,360Updated last week
• wandb / weave
  Weave is a toolkit for developing AI-powered applications, built by Weights & Biases.
  ☆859Updated this week
• stanford-futuredata / ARES
  Automated Evaluation of RAG Systems
  ☆579Updated 3 weeks ago
• mlabonne / llm-autoeval
  Automatically evaluate your LLMs in Google Colab
  ☆615Updated 11 months ago
• relari-ai / continuous-eval
  Data-Driven Evaluation for LLM-Powered Applications
  ☆489Updated 3 months ago
• AnswerDotAI / rerankers
  A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.
  ☆1,384Updated 3 weeks ago
• METR / task-standard
  METR Task Standard
  ☆146Updated 2 months ago
• vec2text / vec2text
  utilities for decoding deep representations (like sentence embeddings) back to text
  ☆795Updated last week
• sierra-research / tau-bench
  Code and Data for Tau-Bench
  ☆437Updated 3 months ago
• huggingface / evaluation-guidebook
  Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard a…
  ☆1,143Updated 3 months ago
• zou-group / textgrad
  TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.
  ☆2,461Updated 3 weeks ago
• lotus-data / lotus
  LOTUS: A semantic query engine for fast and easy LLM-powered data processing
  ☆1,155Updated this week
• potsawee / selfcheckgpt
  SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
  ☆509Updated 9 months ago
• justinchiu / openlogprobs
  Extract full next-token probabilities via language model APIs
  ☆241Updated last year
• AnswerDotAI / RAGatouille
  Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-…
  ☆3,404Updated 2 months ago
• arthur-ai / bench
  A tool for evaluating LLMs
  ☆414Updated 11 months ago
• KarelDO / xmc.dspy
  In-Context Learning for eXtreme Multi-Label Classification (XMC) using only a handful of examples.
  ☆420Updated last year
• MadryLab / context-cite
  Attribute (or cite) statements generated by LLMs back to in-context information.
  ☆228Updated 6 months ago
• mistralai / mistral-common
  ☆713Updated last month
• argilla-io / distilabel
  Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…
  ☆2,642Updated last week
• invariantlabs-ai / invariant
  Guardrails for secure and robust agent development
  ☆200Updated this week
• jxnl / n-levels-of-rag
  ☆194Updated 11 months ago
• datadreamer-dev / DataDreamer
  DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models.   🤖💤
  ☆1,010Updated 2 months ago
• xhluca / bm25s
  Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipy
  ☆1,112Updated this week
• microsoft / sammo
  A library for prompt engineering and optimization (SAMMO = Structure-aware Multi-Objective Metaprompt Optimization)
  ☆666Updated 4 months ago