Alternatives and similar repositories for evaluating-LLMs

Users that are interested in evaluating-LLMs are comparing it to the libraries listed below

• Giskard-AI / awesome-ai-safety
  📚 A curated list of papers & technical articles on AI Quality & Safety
  ☆193Updated 7 months ago
• microsoft / adaptive-testing
  Find and fix bugs in natural language machine learning models using adaptive testing.
  ☆187Updated last year
• NorskRegnesentral / text-anonymization-benchmark
  Annotated corpus + evaluation metrics for text anonymisation
  ☆70Updated 4 months ago
• stanford-crfm / EUAIActJune15
  Stanford CRFM's initiative to assess potential compliance with the draft EU AI Act
  ☆93Updated 2 years ago
• stanford-crfm / fmti
  The Foundation Model Transparency Index
  ☆82Updated last year
• g8a9 / ferret
  A python package for benchmarking interpretability techniques on Transformers.
  ☆214Updated last year
• MantisAI / hugie
  Command Line Interface for Hugging Face Inference Endpoints
  ☆66Updated last year
• mega002 / lm-debugger
  The official code of LM-Debugger, an interactive tool for inspection and intervention in transformer-based language models.
  ☆180Updated 3 years ago
• huggingface / disaggregators
  🤗 Disaggregators: Curated data labelers for in-depth analysis.
  ☆67Updated 2 years ago
• credo-ai / credoai_lens
  Credo AI Lens is a comprehensive assessment framework for AI systems. Lens standardizes model and data assessment, and acts as a central …
  ☆47Updated last year
• flairNLP / fabricator
  [EMNLP 2023 Demo] fabricator - annotating and generating datasets with large language models.
  ☆111Updated last year
• KennethEnevoldsen / augmenty
  Augmenty is an augmentation library based on spaCy for augmenting texts.
  ☆156Updated last year
• infinitylogesh / mutate
  A library to synthesize text datasets using Large Language Models (LLM)
  ☆152Updated 2 years ago
• zeno-ml / zeno
  AI Data Management & Evaluation Platform
  ☆216Updated 2 years ago
• cohere-ai / sandbox-topically
  Topic modeling helpers using managed language models from Cohere. Name text clusters using large GPT models.
  ☆223Updated 2 years ago
• Pacific-AI-Corp / langtest
  Deliver safe & effective language models
  ☆545Updated last month
• webis-de / small-text
  Active Learning for Text Classification in Python
  ☆632Updated 2 weeks ago
• MoritzLaurer / zeroshot-classifier
  Notebooks for training universal 0-shot classifiers on many different tasks
  ☆137Updated 11 months ago
• Data-Provenance-Initiative / Data-Provenance-Collection
  ☆257Updated 8 months ago
• IBM / fastfit
  FastFit ⚡ When LLMs are Unfit Use FastFit ⚡ Fast and Effective Text Classification with Many Classes
  ☆214Updated 2 months ago
• facebookresearch / ResponsibleNLP
  Repository for research in the field of Responsible NLP at Meta.
  ☆202Updated 6 months ago
• salesforce / AuditNLG
  AuditNLG: Auditing Generative AI Language Modeling for Trustworthiness
  ☆101Updated 10 months ago
• allenai / real-toxicity-prompts
  ☆223Updated 4 years ago
• pmbaumgartner / setfit
  ☆43Updated 2 years ago
• dreji18 / Fairness-in-AI
  Detecting Bias and ensuring Fairness in AI solutions
  ☆102Updated 2 years ago
• autonlab / weasel
  Weakly Supervised End-to-End Learning (NeurIPS 2021)
  ☆157Updated 2 years ago
• cohere-ai / magikarp
  Code for the paper "Fishing for Magikarp"
  ☆175Updated 6 months ago
• aiverify-foundation / moonshot-data
  Contains all assets to run with Moonshot Library (Connectors, Datasets and Metrics)
  ☆38Updated 2 months ago
• flairNLP / transformer-ranker
  Efficiently find the best-suited language model (LM) for your NLP task
  ☆132Updated 4 months ago
• argilla-io / adept-augmentations
  A Python library aimed at dissecting and augmenting NER training data.
  ☆59Updated 2 years ago