Alternatives and similar repositories for evaluating-LLMs

Users that are interested in evaluating-LLMs are comparing it to the libraries listed below

• microsoft / adaptive-testing
  Find and fix bugs in natural language machine learning models using adaptive testing.
  ☆185Updated last year
• Giskard-AI / awesome-ai-safety
  📚 A curated list of papers & technical articles on AI Quality & Safety
  ☆192Updated 5 months ago
• flairNLP / fabricator
  [EMNLP 2023 Demo] fabricator - annotating and generating datasets with large language models.
  ☆110Updated last year
• NorskRegnesentral / text-anonymization-benchmark
  Annotated corpus + evaluation metrics for text anonymisation
  ☆64Updated last month
• infinitylogesh / mutate
  A library to synthesize text datasets using Large Language Models (LLM)
  ☆152Updated 2 years ago
• huggingface / disaggregators
  🤗 Disaggregators: Curated data labelers for in-depth analysis.
  ☆67Updated 2 years ago
• facebookresearch / ResponsibleNLP
  Repository for research in the field of Responsible NLP at Meta.
  ☆202Updated 4 months ago
• stanford-crfm / EUAIActJune15
  Stanford CRFM's initiative to assess potential compliance with the draft EU AI Act
  ☆93Updated last year
• mega002 / lm-debugger
  The official code of LM-Debugger, an interactive tool for inspection and intervention in transformer-based language models.
  ☆179Updated 3 years ago
• g8a9 / ferret
  A python package for benchmarking interpretability techniques on Transformers.
  ☆214Updated 11 months ago
• webis-de / small-text
  Active Learning for Text Classification in Python
  ☆623Updated last week
• surge-ai / toxicity
  The world's largest social media toxicity dataset.
  ☆187Updated 3 years ago
• huggingface / that_is_good_data
  ☆65Updated 2 years ago
• zeno-ml / zeno
  AI Data Management & Evaluation Platform
  ☆216Updated last year
• autonlab / weasel
  Weakly Supervised End-to-End Learning (NeurIPS 2021)
  ☆156Updated 2 years ago
• huggingface / olm-datasets
  Pipeline for pulling and processing online language model pretraining data from the web
  ☆177Updated 2 years ago
• argilla-io / adept-augmentations
  A Python library aimed at dissecting and augmenting NER training data.
  ☆58Updated 2 years ago
• cohere-ai / sandbox-topically
  Topic modeling helpers using managed language models from Cohere. Name text clusters using large GPT models.
  ☆223Updated 2 years ago
• salesforce / AuditNLG
  AuditNLG: Auditing Generative AI Language Modeling for Trustworthiness
  ☆102Updated 7 months ago
• bigscience-workshop / lm-evaluation-harness
  A framework for few-shot evaluation of autoregressive language models.
  ☆105Updated 2 years ago
• xaiguy / chippy
  ☆13Updated 2 years ago
• flxst / nerblackbox
  A High-level Library for Named Entity Recognition in Python.
  ☆24Updated last year
• stanford-crfm / fmti
  The Foundation Model Transparency Index
  ☆82Updated last year
• pmbaumgartner / setfit
  ☆43Updated 2 years ago
• timoschick / dino
  This repository contains the code for "Generating Datasets with Pretrained Language Models".
  ☆188Updated 4 years ago
• KennethEnevoldsen / augmenty
  Augmenty is an augmentation library based on spaCy for augmenting texts.
  ☆156Updated last year
• huggingface / large_language_model_training_playbook
  An open collection of implementation tips, tricks and resources for training large language models
  ☆479Updated 2 years ago
• Pacific-AI-Corp / langtest
  Deliver safe & effective language models
  ☆538Updated this week
• MantisAI / hugie
  Command Line Interface for Hugging Face Inference Endpoints
  ☆66Updated last year
• IBM / fastfit
  FastFit ⚡ When LLMs are Unfit Use FastFit ⚡ Fast and Effective Text Classification with Many Classes
  ☆212Updated this week