Alternatives and similar repositories for evaluation-pipeline-2024:

Users that are interested in evaluation-pipeline-2024 are comparing it to the libraries listed below

• GEM-benchmark / GEM-metrics
  Automatic metrics for GEM tasks
  ☆65Updated 2 years ago
• kernelmachine / demix
  DEMix Layers for Modular Language Modeling
  ☆53Updated 3 years ago
• suzgunmirac / crowd-sampling
  Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding
  ☆18Updated 2 years ago
• krishnap25 / mauve-experiments
  ☆38Updated 11 months ago
• lukemelas / mtob
  ☆34Updated 10 months ago
• babylm / evaluation-pipeline-2023
  Evaluation pipeline for the BabyLM Challenge 2023.
  ☆75Updated last year
• kernelmachine / demix-data
  Benchmark API for Multidomain Language Modeling
  ☆24Updated 2 years ago
• peterwestuw / surface-form-competition
  ☆58Updated 2 years ago
• princeton-nlp / ShortcutGrammar
  EMNLP 2022: Finding Dataset Shortcuts with Grammar Induction https://arxiv.org/abs/2210.11560
  ☆58Updated last month
• cindyxinyiwang / expand-via-lexicon-based-adaptation
  Code for ACL 2022 paper "Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation"
  ☆30Updated 3 years ago
• tanyuqian / ctc-gen-eval
  EMNLP 2021 - CTC: A Unified Framework for Evaluating Natural Language Generation
  ☆96Updated 2 years ago
• bigscience-workshop / multilingual-modeling
  BLOOM+1: Adapting BLOOM model to support a new unseen language
  ☆71Updated last year
• leo-liuzy / probe-across-time
  ☆22Updated 3 years ago
• yanaiela / pararel
  ☆44Updated last year
• jkallini / mission-impossible-language-models
  Code repository for the paper "Mission: Impossible Language Models."
  ☆52Updated last week
• ekinakyurek / influence
  Code for "Tracing Knowledge in Language Models Back to the Training Data"
  ☆37Updated 2 years ago
• jzbjyb / lm-calibration
  ☆34Updated 3 years ago
• mohsenfayyaz / GlobEnc
  [NAACL 2022] GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers
  ☆21Updated last year
• machelreid / m2d2
  M2D2: A Massively Multi-domain Language Modeling Dataset (EMNLP 2022) by Machel Reid, Victor Zhong, Suchin Gururangan, Luke Zettlemoyer
  ☆55Updated 2 years ago
• ekinakyurek / lexical
  Paper: Lexicon Learning for Few-Shot Neural Sequence Modeling
  ☆16Updated 3 years ago
• tau-nlp / scrolls
  The official code of EMNLP 2022, "SCROLLS: Standardized CompaRison Over Long Language Sequences".
  ☆69Updated last year
• jennhu / metalinguistic-prompting
  Materials for "Prompting is not a substitute for probability measurements in large language models" (EMNLP 2023)
  ☆23Updated last year
• peterbhase / LAS-NL-Explanations
  Code for paper "Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?"
  ☆21Updated 4 years ago
• awebson / prompt_semantics
  This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”
  ☆85Updated 2 years ago
• nyu-mll / SQuALITY
  Query-focused summarization data
  ☆41Updated 2 years ago
• martiansideofthemoon / longeval-summarization
  Official repository for our EACL 2023 paper "LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization" (https…
  ☆43Updated 8 months ago
• SimengSun / ChapterBreak
  ☆11Updated 10 months ago
• McGill-NLP / FaithDial
  ☆48Updated 2 years ago
• mega002 / ff-layers
  The accompanying code for "Transformer Feed-Forward Layers Are Key-Value Memories". Mor Geva, Roei Schuster, Jonathan Berant, and Omer Le…
  ☆90Updated 3 years ago
• ryokamoi / wice
  This repository contains the dataset and code for "WiCE: Real-World Entailment for Claims in Wikipedia" in EMNLP 2023.
  ☆41Updated last year