bigscience-workshop / lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
☆101Updated last year
Related projects ⓘ
Alternatives and complementary repositories for lm-evaluation-harness
- ☆120Updated 2 months ago
- Tk-Instruct is a Transformer model that is tuned to solve many NLP tasks by following instructions.☆177Updated 2 years ago
- Datasets collection and preprocessings framework for NLP extreme multitask learning☆147Updated 3 months ago
- ☆95Updated last year
- Scalable training for dense retrieval models.☆270Updated last year
- [Data + code] ExpertQA : Expert-Curated Questions and Attributed Answers☆122Updated 7 months ago
- What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets☆188Updated 2 months ago
- ☆166Updated last year
- Token-level Reference-free Hallucination Detection☆92Updated last year
- This project studies the performance and robustness of language models and task-adaptation methods.☆141Updated 5 months ago
- Companion repo for "Evaluating Verifiability in Generative Search Engines".☆80Updated last year
- The official code of EMNLP 2022, "SCROLLS: Standardized CompaRison Over Long Language Sequences".☆68Updated 9 months ago
- ☆73Updated last year
- The LM Contamination Index is a manually created database of contamination evidences for LMs.☆75Updated 6 months ago
- ☆38Updated 6 months ago
- Retrieval as Attention☆82Updated last year
- Code, datasets, and checkpoints for the paper "Improving Passage Retrieval with Zero-Shot Question Generation (EMNLP 2022)"☆93Updated last year
- The official repository for Efficient Long-Text Understanding Using Short-Text Models (Ivgi et al., 2022) paper☆68Updated last year
- Code and Data for "Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering"☆78Updated 2 months ago
- ACL2023 - AlignScore, a metric for factual consistency evaluation.☆110Updated 7 months ago
- Official code and model checkpoints for our EMNLP 2022 paper "RankGen - Improving Text Generation with Large Ranking Models" (https://arx…☆136Updated last year
- ☆66Updated 9 months ago
- Code and data accompanying the paper "TRUE: Re-evaluating Factual Consistency Evaluation".☆71Updated 2 weeks ago
- This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”☆84Updated 2 years ago
- Pretraining Efficiently on S2ORC!☆136Updated 2 weeks ago
- Inquisitive Parrots for Search☆177Updated 8 months ago
- ☆55Updated last year
- The original implementation of Min et al. "Nonparametric Masked Language Modeling" (paper https//arxiv.org/abs/2212.01349)☆156Updated last year
- Code for the arXiv paper: "LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond"☆58Updated 6 months ago
- PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an…☆269Updated 2 years ago