mainlp / awesome-human-label-variationView external linksLinks
A curated list of awesome datasets with human label variation (un-aggregated labels) in Natural Language Processing and Computer Vision, accompanying The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation (EMNLP 2022)
☆98Apr 15, 2024Updated last year
Alternatives and similar repositories for awesome-human-label-variation
Users that are interested in awesome-human-label-variation are comparing it to the libraries listed below
Sorting:
- The official repo for the Dialz Python library - a toolkit for steering vector research.☆21Jul 9, 2025Updated 7 months ago
- CD20200004 from 01/01/2021 to 31/12/2023 - LIG UGA - Python Notebook and Models for the MT Lab @ ALPS 2022☆13Apr 1, 2024Updated last year
- Code for paper "CrossFit : A Few-shot Learning Challenge for Cross-task Generalization in NLP" (https://arxiv.org/abs/2104.08835)☆113Apr 28, 2022Updated 3 years ago
- ☆65Aug 7, 2023Updated 2 years ago
- [ICML'2022] Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network☆21Jul 19, 2022Updated 3 years ago
- A python package to run inference with HuggingFace language and vision-language checkpoints wrapping many convenient features.☆28Sep 14, 2024Updated last year
- ☆21Dec 5, 2022Updated 3 years ago
- ☆10Mar 29, 2021Updated 4 years ago
- POSIX: A Prompt Sensitivity Index for Language Models☆13Nov 13, 2024Updated last year
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Aug 25, 2023Updated 2 years ago
- A framework to train language models to learn invariant representations.☆14Jan 24, 2022Updated 4 years ago
- ☆15Oct 19, 2020Updated 5 years ago
- Pre-processing DBpedia datasets to load into Dgraph☆13Mar 6, 2022Updated 3 years ago
- NeurIPS 2022: Estimating Noise Transition Matrix with Label Correlations for Noisy Multi-Label Learning☆18Mar 3, 2023Updated 2 years ago
- Pretrained Span and span Pair Encoder, code for "Pre-training Entity Relation Encoder with Intra-span and Inter-spanInformation.", EMNLP2…☆18Jan 26, 2022Updated 4 years ago
- A dataset for realistic evaluation of noisy label methods☆14Dec 3, 2023Updated 2 years ago
- Annotation Management for Prodigy, that support multiple users working in many projects☆15Nov 23, 2018Updated 7 years ago
- Resources for paper "DialSummEval: Revisiting summarization evaluation for dialogues"☆15Jul 22, 2025Updated 6 months ago
- This repository includes the masking vocabulary used in the ICLR 2021 spotlight PMI-Masking paper☆14Aug 9, 2021Updated 4 years ago
- [EMNLP 2020] Collective HumAn OpinionS on Natural Language Inference Data☆40Apr 7, 2022Updated 3 years ago
- A Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation, Levy et al., Findings of EMNLP 2021☆14Apr 3, 2022Updated 3 years ago
- The official repo of our research work "Interactive Editing for Text Summarization".☆23Jun 3, 2023Updated 2 years ago
- Code & data for EMNLP 2020 paper "MOCHA: A Dataset for Training and Evaluating Reading Comprehension Metrics".☆16May 3, 2022Updated 3 years ago
- A Package for Cantonese Tokenisation☆18Jun 17, 2021Updated 4 years ago
- Hate speech detection corpus in Korean, shared with EMNLP 2023 paper☆17Apr 19, 2024Updated last year
- ☆15Apr 10, 2018Updated 7 years ago
- Overview of corpora/datasets for Germanic low-resource languages and dialects. Accompanies "A Survey of Corpora for Germanic Low-Resource…☆27Updated this week
- CrossRE: A Cross-Domain Dataset for Relation Extraction (Findings of EMNLP 2022)☆49Aug 20, 2024Updated last year
- Automatically detect errors in annotated corpora.☆48Sep 8, 2023Updated 2 years ago
- My NER Experiments with ModernBERT and Ettin☆26Jul 17, 2025Updated 6 months ago
- Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"☆59Jan 12, 2023Updated 3 years ago
- Hugging Face Download (Cache) Manager☆21Aug 7, 2022Updated 3 years ago
- Code for paper "Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs"☆28May 2, 2022Updated 3 years ago
- Code & Data for the paper "RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models"☆32May 31, 2021Updated 4 years ago
- Streamlit demo app to demonstrate the features of transformers interpret with multiple models.☆25Jun 24, 2021Updated 4 years ago
- Slides from previous talks.☆29Nov 23, 2023Updated 2 years ago
- Ranking of fine-tuned HF models as base models.☆36Sep 17, 2025Updated 4 months ago
- A Pytorch Implementations for Various Vector Quantization Methods☆33Sep 14, 2021Updated 4 years ago
- Pre-trained ELECTRA from Hong Kong data☆29Jul 7, 2020Updated 5 years ago