google-research-datasets / AIS
AIS is an evaluation framework for assessing whether the output of natural language models only contains information about the external world that is verifiable in source documents, or "Attributable to Identified Sources".
☆31Updated 2 years ago
Alternatives and similar repositories for AIS:
Users that are interested in AIS are comparing it to the libraries listed below
- Official repository for our EACL 2023 paper "LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization" (https…☆43Updated 6 months ago
- Code and pre-trained models for "ReasonBert: Pre-trained to Reason with Distant Supervision", EMNLP'2021☆29Updated 2 years ago
- Resources for the shared task on conversational question answering SCAI-QReCC 2021☆27Updated 2 years ago
- ☆38Updated last year
- ☆33Updated last year
- ☆97Updated 2 years ago
- ☆47Updated 2 years ago
- Code for paper "Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?"☆21Updated 4 years ago
- ☆15Updated 3 years ago
- Query-focused summarization data☆41Updated 2 years ago
- This repository contains the dataset and code for "WiCE: Real-World Entailment for Claims in Wikipedia" in EMNLP 2023.☆41Updated last year
- This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”☆85Updated 2 years ago
- PyTorch code for "FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization" (NAACL 2022)☆38Updated 2 years ago
- ☆58Updated 2 years ago
- TBC☆26Updated 2 years ago
- Detect hallucinated tokens for conditional sequence generation.☆64Updated 2 years ago
- Companion repo for "Evaluating Verifiability in Generative Search Engines".☆83Updated last year
- Efficient Memory-Augmented Transformers☆34Updated 2 years ago
- Code & data for EMNLP 2020 paper "MOCHA: A Dataset for Training and Evaluating Reading Comprehension Metrics".☆16Updated 2 years ago
- Code, data, and pretrained models for the paper "Generating Wikipedia Article Sections from Diverse Data Sources"☆20Updated 4 years ago
- The official implemetation of "Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks" (NAACL 2022).☆43Updated 2 years ago
- ☆11Updated 8 months ago
- Apps built using Inspired Cognition's Critique.☆58Updated 2 years ago
- Code, datasets, models for the paper "Automatic Evaluation of Attribution by Large Language Models"☆54Updated last year
- Code for Massive-scale Decoding for Text Generation using Lattices☆45Updated 2 years ago
- ☆22Updated 2 years ago
- Code and data accompanying the paper "TRUE: Re-evaluating Factual Consistency Evaluation".☆75Updated 3 months ago
- Easy-to-use MIRAGE code for faithful answer attribution in RAG applications. Paper: https://aclanthology.org/2024.emnlp-main.347/☆21Updated 2 months ago
- Contrastive Fact Verification☆71Updated 2 years ago
- Code for the arXiv paper: "LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond"☆59Updated last month