A curated list of awesome datasets with human label variation (un-aggregated labels) in Natural Language Processing and Computer Vision, accompanying The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation (EMNLP 2022)
☆99Apr 15, 2024Updated last year
Alternatives and similar repositories for awesome-human-label-variation
Users that are interested in awesome-human-label-variation are comparing it to the libraries listed below
Sorting:
- CD20200004 from 01/01/2021 to 31/12/2023 - LIG UGA - Python Notebook and Models for the MT Lab @ ALPS 2022☆13Apr 1, 2024Updated last year
- Code for paper "CrossFit : A Few-shot Learning Challenge for Cross-task Generalization in NLP" (https://arxiv.org/abs/2104.08835)☆113Apr 28, 2022Updated 3 years ago
- ☆65Aug 7, 2023Updated 2 years ago
- [ICML'2022] Estimating Instance-dependent Bayes-label Transition Matrix using a Deep Neural Network☆21Jul 19, 2022Updated 3 years ago
- A python package to run inference with HuggingFace language and vision-language checkpoints wrapping many convenient features.☆28Sep 14, 2024Updated last year
- ☆10Mar 29, 2021Updated 4 years ago
- POSIX: A Prompt Sensitivity Index for Language Models☆13Nov 13, 2024Updated last year
- ☆21Dec 5, 2022Updated 3 years ago
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Aug 25, 2023Updated 2 years ago
- ☆14Jun 8, 2018Updated 7 years ago
- Pre-processing DBpedia datasets to load into Dgraph☆13Mar 6, 2022Updated 4 years ago
- NeurIPS 2022: Estimating Noise Transition Matrix with Label Correlations for Noisy Multi-Label Learning☆18Mar 3, 2023Updated 3 years ago
- A dataset for realistic evaluation of noisy label methods☆14Dec 3, 2023Updated 2 years ago
- Pretrained Span and span Pair Encoder, code for "Pre-training Entity Relation Encoder with Intra-span and Inter-spanInformation.", EMNLP2…☆18Jan 26, 2022Updated 4 years ago
- This repository includes the masking vocabulary used in the ICLR 2021 spotlight PMI-Masking paper☆14Aug 9, 2021Updated 4 years ago
- [EMNLP 2020] Collective HumAn OpinionS on Natural Language Inference Data☆40Apr 7, 2022Updated 3 years ago
- A Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation, Levy et al., Findings of EMNLP 2021☆14Apr 3, 2022Updated 3 years ago
- The official repo of our research work "Interactive Editing for Text Summarization".☆23Jun 3, 2023Updated 2 years ago
- Code & data for EMNLP 2020 paper "MOCHA: A Dataset for Training and Evaluating Reading Comprehension Metrics".☆16May 3, 2022Updated 3 years ago
- Hate speech detection corpus in Korean, shared with EMNLP 2023 paper☆17Apr 19, 2024Updated last year
- A Package for Cantonese Tokenisation☆18Jun 17, 2021Updated 4 years ago
- PyTorch and NNsight implementation of AtP* (Kramar et al 2024, DeepMind)☆20Jan 19, 2025Updated last year
- ☆15Apr 10, 2018Updated 7 years ago
- CrossRE: A Cross-Domain Dataset for Relation Extraction (Findings of EMNLP 2022)☆49Aug 20, 2024Updated last year
- My NER Experiments with ModernBERT and Ettin☆26Jul 17, 2025Updated 7 months ago
- Code for paper "Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs"☆28May 2, 2022Updated 3 years ago
- Hugging Face Download (Cache) Manager☆22Aug 7, 2022Updated 3 years ago
- Streamlit demo app to demonstrate the features of transformers interpret with multiple models.☆25Jun 24, 2021Updated 4 years ago
- ☆27Dec 23, 2023Updated 2 years ago
- ☆62Apr 17, 2024Updated last year
- Slides from previous talks.☆29Nov 23, 2023Updated 2 years ago
- Ranking of fine-tuned HF models as base models.☆36Sep 17, 2025Updated 5 months ago
- A Pytorch Implementations for Various Vector Quantization Methods☆34Sep 14, 2021Updated 4 years ago
- Benchmarking algorithms for assessing quality of data labeled by multiple annotators☆34Dec 3, 2025Updated 3 months ago
- Official repo for "SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-Supervised Learning", accepted by ICLR 2023.☆22Jan 31, 2023Updated 3 years ago
- A categorical archive of ChatGPT failures☆64May 25, 2023Updated 2 years ago
- The official implementation for ACL 2021 "Challenges in Information Seeking QA: Unanswerable Questions and Paragraph Retrieval".☆28Jun 19, 2021Updated 4 years ago
- ☆13Oct 5, 2025Updated 5 months ago
- This repo contains code to reproduce some of the results presented in the paper "SentenceMIM: A Latent Variable Language Model"☆28Jun 22, 2022Updated 3 years ago