asaparov / prontoqa
Synthetic question-answering dataset to formally analyze the chain-of-thought output of large language models on a reasoning task.
☆144Updated 6 months ago
Alternatives and similar repositories for prontoqa:
Users that are interested in prontoqa are comparing it to the libraries listed below
- ☆82Updated 8 months ago
- [ACL'24] Code and data of paper "When is Tree Search Useful for LLM Planning? It Depends on the Discriminator"☆54Updated last year
- Inspecting and Editing Knowledge Representations in Language Models☆115Updated last year
- How do transformer LMs encode relations?☆46Updated last year
- 🔗 LINC: Logical Inference via Neurosymbolic Computation [EMNLP2023]☆66Updated last year
- ☆93Updated last year
- Code for paper "LEVER: Learning to Verifiy Language-to-Code Generation with Execution" (ICML'23)☆86Updated last year
- Github repository for "FELM: Benchmarking Factuality Evaluation of Large Language Models" (NeurIPS 2023)☆58Updated last year
- ☆95Updated last year
- ☆53Updated last year
- ☆115Updated 9 months ago
- [EMNLP 2023] MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions☆108Updated 7 months ago
- ☆43Updated 8 months ago
- ☆83Updated 2 months ago
- A library for efficient patching and automatic circuit discovery.☆62Updated 2 months ago
- Code for the paper <SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning>☆49Updated last year
- Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision☆120Updated 7 months ago
- [ICLR'24 Spotlight] "Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts"☆67Updated last year
- Official Repo for ICLR 2024 paper MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback by Xingyao Wang*, Ziha…☆122Updated 10 months ago
- the instructions and demonstrations for building a formal logical reasoning capable GLM☆53Updated 7 months ago
- The official repo for "TheoremQA: A Theorem-driven Question Answering dataset" (EMNLP 2023)☆31Updated 11 months ago
- [ICLR 2024] Evaluating Large Language Models at Evaluating Instruction Following☆124Updated 9 months ago
- For OpenMOSS Mechanistic Interpretability Team's Sparse Autoencoder (SAE) research.☆109Updated 2 weeks ago
- Official Code Repository for LM-Steer Paper: "Word Embeddings Are Steers for Language Models" (ACL 2024 Outstanding Paper Award)☆99Updated 6 months ago
- Awesome LLM Self-Consistency: a curated list of Self-consistency in Large Language Models☆96Updated 8 months ago
- Code release for "Debating with More Persuasive LLMs Leads to More Truthful Answers"☆103Updated last year
- Repo accompanying our paper "Do Llamas Work in English? On the Latent Language of Multilingual Transformers".☆73Updated last year
- datasets from the paper "Towards Understanding Sycophancy in Language Models"☆74Updated last year
- A dataset of LLM-generated chain-of-thought steps annotated with mistake location.☆80Updated 8 months ago
- The accompanying code for "Transformer Feed-Forward Layers Are Key-Value Memories". Mor Geva, Roei Schuster, Jonathan Berant, and Omer Le…☆90Updated 3 years ago