bigscience-workshop / lm-evaluation-harnessLinks
A framework for few-shot evaluation of autoregressive language models.
☆105Updated 2 years ago
Alternatives and similar repositories for lm-evaluation-harness
Users that are interested in lm-evaluation-harness are comparing it to the libraries listed below
Sorting:
- Tk-Instruct is a Transformer model that is tuned to solve many NLP tasks by following instructions.☆181Updated 2 years ago
- ☆138Updated 6 months ago
- Scalable training for dense retrieval models.☆299Updated last month
- This project studies the performance and robustness of language models and task-adaptation methods.☆150Updated last year
- [Data + code] ExpertQA : Expert-Curated Questions and Attributed Answers☆131Updated last year
- Token-level Reference-free Hallucination Detection☆95Updated 2 years ago
- What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets☆223Updated 8 months ago
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆208Updated last year
- ☆48Updated last year
- Code and model release for the paper "Task-aware Retrieval with Instructions" by Asai et al.☆163Updated last year
- Code for the arXiv paper: "LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond"☆59Updated 6 months ago
- The official repository for Efficient Long-Text Understanding Using Short-Text Models (Ivgi et al., 2022) paper☆69Updated 2 years ago
- Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)☆462Updated 2 years ago
- Code and data accompanying the paper "TRUE: Re-evaluating Factual Consistency Evaluation".☆81Updated 3 weeks ago
- A Multilingual Replicable Instruction-Following Model☆94Updated 2 years ago
- ☆183Updated last month
- ☆100Updated 2 years ago
- ☆180Updated 2 years ago
- Datasets collection and preprocessings framework for NLP extreme multitask learning☆185Updated 3 weeks ago
- The official code of LM-Debugger, an interactive tool for inspection and intervention in transformer-based language models.☆178Updated 3 years ago
- Code and Data for "Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering"☆86Updated 11 months ago
- Repository for EMNLP 2022 Paper: Towards a Unified Multi-Dimensional Evaluator for Text Generation☆206Updated last year
- DSIR large-scale data selection framework for language model training☆257Updated last year
- PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an…☆276Updated 2 years ago
- ☆72Updated 2 years ago
- The official code of EMNLP 2022, "SCROLLS: Standardized CompaRison Over Long Language Sequences".☆70Updated last year
- Interpreting Language Models with Contrastive Explanations (EMNLP 2022 Best Paper Honorable Mention)☆62Updated 3 years ago
- This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”☆85Updated 3 years ago
- Finetune mistral-7b-instruct for sentence embeddings☆85Updated last year
- [AAAI 2024] Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following☆79Updated 10 months ago