Alternatives and similar repositories for evaluating-LLMs

Users that are interested in evaluating-LLMs are comparing it to the libraries listed below

• Giskard-AI / awesome-ai-safety
  📚 A curated list of papers & technical articles on AI Quality & Safety
  ☆195Updated 8 months ago
• NorskRegnesentral / text-anonymization-benchmark
  Annotated corpus + evaluation metrics for text anonymisation
  ☆70Updated 5 months ago
• stanford-crfm / EUAIActJune15
  Stanford CRFM's initiative to assess potential compliance with the draft EU AI Act
  ☆93Updated 2 years ago
• microsoft / adaptive-testing
  Find and fix bugs in natural language machine learning models using adaptive testing.
  ☆188Updated last year
• g8a9 / ferret
  A python package for benchmarking interpretability techniques on Transformers.
  ☆214Updated last year
• IBM / ensemble-instruct
  codebase release for EMNLP2023 paper publication
  ☆19Updated 3 months ago
• MantisAI / hugie
  Command Line Interface for Hugging Face Inference Endpoints
  ☆66Updated last year
• allenai / real-toxicity-prompts
  ☆226Updated 4 years ago
• zeno-ml / zeno
  AI Data Management & Evaluation Platform
  ☆216Updated 2 years ago
• facebookresearch / ResponsibleNLP
  Repository for research in the field of Responsible NLP at Meta.
  ☆204Updated 7 months ago
• stanford-crfm / fmti
  The Foundation Model Transparency Index
  ☆84Updated 3 weeks ago
• microsoft / TOXIGEN
  This repo contains the code for generating the ToxiGen dataset, published at ACL 2022.
  ☆345Updated last year
• salesforce / AuditNLG
  AuditNLG: Auditing Generative AI Language Modeling for Trustworthiness
  ☆101Updated 11 months ago
• argilla-io / adept-augmentations
  A Python library aimed at dissecting and augmenting NER training data.
  ☆59Updated 2 years ago
• flairNLP / fabricator
  [EMNLP 2023 Demo] fabricator - annotating and generating datasets with large language models.
  ☆111Updated last year
• microsoft / presidio-research
  This package features data-science related tasks for developing new recognizers for Presidio. It is used for the evaluation of the entire…
  ☆253Updated 2 weeks ago
• fiddler-labs / fiddler-auditor
  Fiddler Auditor is a tool to evaluate language models.
  ☆188Updated last year
• mega002 / lm-debugger
  The official code of LM-Debugger, an interactive tool for inspection and intervention in transformer-based language models.
  ☆180Updated 3 years ago
• mlcommons / modelbench
  Run safety benchmarks against AI models and view detailed reports showing how well they performed.
  ☆114Updated last week
• cohere-ai / magikarp
  Code for the paper "Fishing for Magikarp"
  ☆177Updated 7 months ago
• EdyVision / pii-codex
  A research python package for detecting, categorizing, and assessing the severity of personal identifiable information (PII)
  ☆94Updated last week
• huggingface / disaggregators
  🤗 Disaggregators: Curated data labelers for in-depth analysis.
  ☆67Updated 2 years ago
• aiverify-foundation / aiverify
  AI Verify
  ☆39Updated last week
• aiverify-foundation / moonshot-data
  Contains all assets to run with Moonshot Library (Connectors, Datasets and Metrics)
  ☆39Updated last week
• bigscience-workshop / lm-evaluation-harness
  A framework for few-shot evaluation of autoregressive language models.
  ☆105Updated 2 years ago
• infinitylogesh / mutate
  A library to synthesize text datasets using Large Language Models (LLM)
  ☆152Updated 2 years ago
• Pacific-AI-Corp / langtest
  Deliver safe & effective language models
  ☆548Updated this week
• Data-Provenance-Initiative / Data-Provenance-Collection
  ☆260Updated 9 months ago
• UKPLab / gpl
  Powerful unsupervised domain adaptation method for dense retrieval. Requires only unlabeled corpus and yields massive improvement: "GPL: …
  ☆338Updated 2 years ago
• credo-ai / credoai_lens
  Credo AI Lens is a comprehensive assessment framework for AI systems. Lens standardizes model and data assessment, and acts as a central …
  ☆47Updated last year