bigscience-workshop / lm-evaluation-harness
A framework for few-shot evaluation of autoregressive language models.
☆101Updated last year
Related projects ⓘ
Alternatives and complementary repositories for lm-evaluation-harness
- ☆122Updated 2 months ago
- Tk-Instruct is a Transformer model that is tuned to solve many NLP tasks by following instructions.☆177Updated 2 years ago
- This project studies the performance and robustness of language models and task-adaptation methods.☆141Updated 6 months ago
- [Data + code] ExpertQA : Expert-Curated Questions and Attributed Answers☆122Updated 8 months ago
- Datasets collection and preprocessings framework for NLP extreme multitask learning☆149Updated 4 months ago
- Scalable training for dense retrieval models.☆271Updated last year
- This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”☆84Updated 2 years ago
- Official code and model checkpoints for our EMNLP 2022 paper "RankGen - Improving Text Generation with Large Ranking Models" (https://arx…☆136Updated last year
- Companion repo for "Evaluating Verifiability in Generative Search Engines".☆81Updated last year
- The official code of EMNLP 2022, "SCROLLS: Standardized CompaRison Over Long Language Sequences".☆68Updated 10 months ago
- ☆95Updated last year
- Token-level Reference-free Hallucination Detection☆93Updated last year
- ☆167Updated last year
- What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets☆190Updated this week
- ☆73Updated last year
- ☆80Updated 2 years ago
- [AAAI 2024] Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following☆79Updated 2 months ago
- Repo for the paper "Large Language Models Struggle to Learn Long-Tail Knowledge"☆73Updated last year
- PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an…☆269Updated 2 years ago
- ☆55Updated last year
- Inspecting and Editing Knowledge Representations in Language Models☆108Updated last year
- Code and model release for the paper "Task-aware Retrieval with Instructions" by Asai et al.☆160Updated last year
- Inquisitive Parrots for Search☆178Updated 8 months ago
- ☆158Updated last year
- Dense hybrid representations for text retrieval☆62Updated last year
- ☆67Updated 9 months ago
- [ICLR 2023] Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners☆111Updated 2 months ago
- Retrieval as Attention☆83Updated last year
- The official repository for Efficient Long-Text Understanding Using Short-Text Models (Ivgi et al., 2022) paper☆67Updated last year
- Code and Data for "Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering"☆78Updated 3 months ago