Alternatives and similar repositories for evaluating-LLMs

Users that are interested in evaluating-LLMs are comparing it to the libraries listed below

• microsoft / adaptive-testing
  Find and fix bugs in natural language machine learning models using adaptive testing.
  ☆187Updated last year
• Giskard-AI / awesome-ai-safety
  📚 A curated list of papers & technical articles on AI Quality & Safety
  ☆193Updated 6 months ago
• fiddler-labs / fiddler-auditor
  Fiddler Auditor is a tool to evaluate language models.
  ☆188Updated last year
• stanford-crfm / EUAIActJune15
  Stanford CRFM's initiative to assess potential compliance with the draft EU AI Act
  ☆93Updated 2 years ago
• zeno-ml / zeno
  AI Data Management & Evaluation Platform
  ☆216Updated 2 years ago
• stanford-crfm / fmti
  The Foundation Model Transparency Index
  ☆83Updated last year
• salesforce / AuditNLG
  AuditNLG: Auditing Generative AI Language Modeling for Trustworthiness
  ☆101Updated 9 months ago
• h2oai / h2o-LLM-eval
  Large-language Model Evaluation framework with Elo Leaderboard and A-B testing
  ☆52Updated last year
• huggingface / disaggregators
  🤗 Disaggregators: Curated data labelers for in-depth analysis.
  ☆67Updated 2 years ago
• g8a9 / ferret
  A python package for benchmarking interpretability techniques on Transformers.
  ☆212Updated last year
• microsoft / TOXIGEN
  This repo contains the code for generating the ToxiGen dataset, published at ACL 2022.
  ☆337Updated last year
• IBM / ensemble-instruct
  codebase release for EMNLP2023 paper publication
  ☆19Updated last month
• flairNLP / fabricator
  [EMNLP 2023 Demo] fabricator - annotating and generating datasets with large language models.
  ☆110Updated last year
• openai / moderation-api-release
  ☆149Updated 3 years ago
• mlcommons / modelbench
  Run safety benchmarks against AI models and view detailed reports showing how well they performed.
  ☆108Updated this week
• allenai / real-toxicity-prompts
  ☆221Updated 4 years ago
• MantisAI / hugie
  Command Line Interface for Hugging Face Inference Endpoints
  ☆66Updated last year
• UKPLab / gpl
  Powerful unsupervised domain adaptation method for dense retrieval. Requires only unlabeled corpus and yields massive improvement: "GPL: …
  ☆338Updated 2 years ago
• credo-ai / credoai_lens
  Credo AI Lens is a comprehensive assessment framework for AI systems. Lens standardizes model and data assessment, and acts as a central …
  ☆47Updated last year
• JoaoLages / RATransformers
  RATransformers 🐭- Make your transformer (like BERT, RoBERTa, GPT-2 and T5) Relation Aware!
  ☆41Updated 2 years ago
• sileod / tasksource
  Datasets collection and preprocessings framework for NLP extreme multitask learning
  ☆188Updated 4 months ago
• coastalcph / lex-glue
  LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
  ☆228Updated 3 months ago
• MoritzLaurer / zeroshot-classifier
  Notebooks for training universal 0-shot classifiers on many different tasks
  ☆136Updated 10 months ago
• microsoft / Multilingual-Evaluation-of-Generative-AI-MEGA
  Code for Multilingual Eval of Generative AI paper published at EMNLP 2023
  ☆70Updated last year
• surge-ai / toxicity
  The world's largest social media toxicity dataset.
  ☆187Updated 3 years ago
• bigscience-workshop / lm-evaluation-harness
  A framework for few-shot evaluation of autoregressive language models.
  ☆104Updated 2 years ago
• facebookresearch / ResponsibleNLP
  Repository for research in the field of Responsible NLP at Meta.
  ☆202Updated 5 months ago
• anthropics / evals
  ☆305Updated last year
• NorskRegnesentral / text-anonymization-benchmark
  Annotated corpus + evaluation metrics for text anonymisation
  ☆70Updated 3 months ago
• webis-de / small-text
  Active Learning for Text Classification in Python
  ☆631Updated last week