Alternatives and similar repositories for evaluation-pipeline-2024

Users that are interested in evaluation-pipeline-2024 are comparing it to the libraries listed below

• babylm / evaluation-pipeline-2023
  Evaluation pipeline for the BabyLM Challenge 2023.
  ☆76Updated last year
• GEM-benchmark / GEM-metrics
  Automatic metrics for GEM tasks
  ☆66Updated 2 years ago
• lukemelas / mtob
  ☆36Updated last year
• mega002 / ff-layers
  The accompanying code for "Transformer Feed-Forward Layers Are Key-Value Memories". Mor Geva, Roei Schuster, Jonathan Berant, and Omer Le…
  ☆94Updated 3 years ago
• tau-nlp / scrolls
  The official code of EMNLP 2022, "SCROLLS: Standardized CompaRison Over Long Language Sequences".
  ☆70Updated last year
• awebson / prompt_semantics
  This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”
  ☆85Updated 3 years ago
• kernelmachine / demix-data
  Benchmark API for Multidomain Language Modeling
  ☆25Updated 2 years ago
• qkaren / COLD_decoding
  ☆108Updated 3 years ago
• machelreid / m2d2
  M2D2: A Massively Multi-domain Language Modeling Dataset (EMNLP 2022) by Machel Reid, Victor Zhong, Suchin Gururangan, Luke Zettlemoyer
  ☆54Updated 2 years ago
• kernelmachine / demix
  DEMix Layers for Modular Language Modeling
  ☆53Updated 3 years ago
• mt-upc / transformer-contributions
  Measuring the Mixing of Contextual Information in the Transformer
  ☆30Updated 2 years ago
• hsajjad / Interpretability-Tutorial-NAACL2021
  ☆24Updated 4 years ago
• ekinakyurek / influence
  Code for "Tracing Knowledge in Language Models Back to the Training Data"
  ☆38Updated 2 years ago
• nyu-mll / SQuALITY
  Query-focused summarization data
  ☆42Updated 2 years ago
• neulab / knn-transformers
  PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an…
  ☆275Updated 2 years ago
• ZurichNLP / mbr
  Minimum Bayes Risk Decoding for Hugging Face Transformers
  ☆58Updated last year
• EleutherAI / knowledge-neurons
  A library for finding knowledge neurons in pretrained transformer models.
  ☆158Updated 3 years ago
• yanaiela / pararel
  ☆45Updated last year
• INK-USC / CrossFit
  Code for paper "CrossFit : A Few-shot Learning Challenge for Cross-task Generalization in NLP" (https://arxiv.org/abs/2104.08835)
  ☆111Updated 3 years ago
• keyonvafa / sequential-rationales
  Rationales for Sequential Predictions
  ☆40Updated 3 years ago
• mohsenfayyaz / GlobEnc
  [NAACL 2022] GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers
  ☆21Updated 2 years ago
• peterwestuw / surface-form-competition
  ☆58Updated 3 years ago
• nyu-mll / quality
  ☆137Updated 6 months ago
• gorokoba560 / norm-analysis-of-transformer
  ☆86Updated last year
• mt-upc / transformer-contributions-nmt
  ☆17Updated 2 years ago
• suzgunmirac / crowd-sampling
  Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding
  ☆18Updated 2 years ago
• timoschick / self-debiasing
  This repository contains the code for "Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP".
  ☆88Updated 3 years ago
• cpllab / syntactic-generalization
  Code and data for "A Systematic Assessment of Syntactic Generalization in Neural Language Models"
  ☆28Updated 4 years ago
• neulab / retomaton
  PyTorch code for the RetoMaton paper: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022)
  ☆73Updated 3 years ago
• McGill-NLP / FaithDial
  ☆48Updated 2 years ago