psunlpgroup / ReaLMistakeLinks
This repository includes a benchmark and code for the paper "Evaluating LLMs at Detecting Errors in LLM Responses".
☆30Updated last year
Alternatives and similar repositories for ReaLMistake
Users that are interested in ReaLMistake are comparing it to the libraries listed below
Sorting:
- [ACL'24] Code and data of paper "When is Tree Search Useful for LLM Planning? It Depends on the Discriminator"☆54Updated last year
- Scalable Meta-Evaluation of LLMs as Evaluators☆43Updated last year
- ☆32Updated last year
- ☆44Updated last year
- [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training☆23Updated last year
- Repo accompanying our paper "Do Llamas Work in English? On the Latent Language of Multilingual Transformers".☆80Updated last year
- PASTA: Post-hoc Attention Steering for LLMs☆132Updated last year
- ☆75Updated last year
- Grade-School Math with Irrelevant Context (GSM-IC) benchmark is an arithmetic reasoning dataset built upon GSM8K, by adding irrelevant se…☆65Updated 2 years ago
- A dataset of LLM-generated chain-of-thought steps annotated with mistake location.☆84Updated last year
- the instructions and demonstrations for building a formal logical reasoning capable GLM☆55Updated last year
- Evaluate the Quality of Critique☆36Updated last year
- ☆56Updated last year
- This repository contains data, code and models for contextual noncompliance.☆24Updated last year
- [ICLR 2024] Evaluating Large Language Models at Evaluating Instruction Following☆134Updated last year
- Source code of "Reasons to Reject? Aligning Language Models with Judgments"☆58Updated last year
- ☆47Updated last year
- A simple GPT-based evaluation tool for multi-aspect, interpretable assessment of LLMs.☆89Updated last year
- [EMNLP'24] LongHeads: Multi-Head Attention is Secretly a Long Context Processor☆31Updated last year
- Code for the 2025 ACL publication "Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs"☆33Updated 5 months ago
- An open-source library for contamination detection in NLP datasets and Large Language Models (LLMs).☆58Updated last year
- Implementation of the paper: "Answering Questions by Meta-Reasoning over Multiple Chains of Thought"☆96Updated last year
- Code and data for paper "Context-faithful Prompting for Large Language Models".☆41Updated 2 years ago
- ☆50Updated 2 years ago
- Contrastive Chain-of-Thought Prompting☆68Updated 2 years ago
- Exploring the Limitations of Large Language Models on Multi-Hop Queries☆29Updated 9 months ago
- Lightweight tool to identify Data Contamination in LLMs evaluation☆53Updated last year
- Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators (Liu et al.; COLM 2024)☆48Updated 11 months ago
- ☆15Updated last year
- Github repository for "FELM: Benchmarking Factuality Evaluation of Large Language Models" (NeurIPS 2023)☆62Updated last year