facebookresearch / irt-leaderboard
Leaderboards are widely used in NLP and push the field forward. While leaderboards are a straightforward ranking of NLP models, this simplicity can mask nuances in evaluation items (examples) and subjects (NLP models). Rather than replace leaderboards, we advocate a re-imagining so that they better highlight if and where progress is made. Buildi…
☆17Updated 3 years ago
Alternatives and similar repositories for irt-leaderboard:
Users that are interested in irt-leaderboard are comparing it to the libraries listed below
- Bayesian Assessment of Hypotheses☆24Updated last year
- ☆38Updated 4 years ago
- ☆24Updated 5 years ago
- Python code for training models in the ACL paper, "Simple and Effective Paraphrastic Similarity from Parallel Translations".☆22Updated 5 years ago
- This repository contains the code for the Form-Context Model and its Attentive Mimicking variant.☆31Updated 4 years ago
- ☆19Updated 5 years ago
- Factorization of the neural parameter space for zero-shot multi-lingual and multi-task transfer☆39Updated 4 years ago
- Author implementation of the paper "Don’t paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing"☆20Updated 4 years ago
- Code for our ACL '20 paper "Representation Engineering with Natural Language Explanations"☆29Updated 4 years ago
- ☆12Updated 4 years ago
- Diverse Natural Language Inference Collection - NLI dataset that can used to evaluate how well models perform distinct types of reasoning…☆36Updated 4 years ago
- Cross-lingual TRansfer Evaluation of Multilingual Encoders (XTREME)☆22Updated 5 years ago
- ☆16Updated 6 years ago
- Statistics on multilingual datasets☆17Updated 2 years ago
- ☆17Updated last year
- Defeasible Natural Language Inference☆12Updated 4 years ago
- Relevant code for the "Show Your Work" paper, EMNLP 2019.☆18Updated 5 years ago
- Question-Answer Meaning Representation☆48Updated 3 years ago
- ☆29Updated last year
- Converter from UD-trees to BART representation☆36Updated last year
- Syntactic evaluation sets, attribute-varying grammars, and code for replicating the CLAMS paper. ACL 2020.☆16Updated 5 months ago
- The dataset and statistical analysis code released with the submission of EMNLP 2017 paper "Why We Need New Evaluation Metrics for NLG"☆19Updated 3 years ago
- Efficient Sentence Embedding via Semantic Subspace Analysis☆14Updated 5 years ago
- Code for Massive-scale Decoding for Text Generation using Lattices☆44Updated 2 years ago
- Implementation of "Effective Adversarial Regularization for Neural Machine Translation", ACL 2019☆21Updated 5 years ago
- ☆27Updated last year
- This repository hosts the code for a tokenizer of tweets.☆12Updated 6 years ago
- Code for the paper "Latent Relation Language Models" at AAAI-20.☆41Updated 4 years ago
- codebase for the Text-based NP Enrichment (TNE) paper☆20Updated last year
- Post-editing Datasets by Rakuten (PEDRa)☆14Updated 3 years ago