Alternatives and similar repositories for counterfactual-evaluation:

Users that are interested in counterfactual-evaluation are comparing it to the libraries listed below

• GAIR-NLP / alignment-for-honesty
  ☆73Updated 11 months ago
• cicl-stanford / procedural-evals-tom
  ☆31Updated last year
• YuxiXie / SelfEval-Guided-Decoding
  ☆95Updated last year
• princeton-nlp / unintentional-unalignment
  [ICLR 2025] Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
  ☆25Updated 3 months ago
• Re-Align / AlignTDS
  Analyzing LLM Alignment via Token distribution shift
  ☆16Updated last year
• qtli / GSM-Plus
  GSM-Plus: Data, Code, and Evaluation for Enhancing Robust Mathematical Reasoning in Math Word Problems.
  ☆60Updated 10 months ago
• yuzhaouoe / SAE-based-representation-engineering
  [NAACL'25 Oral] Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering
  ☆54Updated 5 months ago
• McGill-NLP / polytropon
  ☆53Updated 2 years ago
• Shentao-YANG / Preference_Grounded_Guidance
  Source codes for "Preference-grounded Token-level Guidance for Language Model Fine-tuning" (NeurIPS 2023).
  ☆16Updated 4 months ago
• archiki / ReCEval
  Supporting code for ReCEval paper
  ☆28Updated 7 months ago
• sylinrl / CalibratedMath
  Teaching Models to Express Their Uncertainty in Words
  ☆38Updated 2 years ago
• ruiqi-zhong / nlparam
  Augmenting Statistical Models with Natural Language Parameters
  ☆26Updated 7 months ago
• Edward-Sun / easy-to-hard
  Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision
  ☆120Updated 7 months ago
• google-research-datasets / GSM-IC
  Grade-School Math with Irrelevant Context (GSM-IC) benchmark is an arithmetic reasoning dataset built upon GSM8K, by adding irrelevant se…
  ☆60Updated 2 years ago
• dannyallover / overthinking_the_truth
  ☆29Updated last year
• edenbiran / HoppingTooLate
  Exploring the Limitations of Large Language Models on Multi-Hop Queries
  ☆25Updated 2 months ago
• GAIR-NLP / Preference-Dissection
  ☆25Updated 11 months ago
• ars22 / scaling-LLM-math-synthetic-data
  Code and data used in the paper: "Training on Incorrect Synthetic Data via RL Scales LLM Math Reasoning Eight-Fold"
  ☆30Updated 10 months ago
• FreedomIntelligence / OVM
  ☆66Updated last year
• matt-seb-ho / WikiWhy
  WikiWhy is a new benchmark for evaluating LLMs' ability to explain between cause-effect relationships. It is a QA dataset containing 9000…
  ☆47Updated last year
• roeehendel / icl_task_vectors
  ☆93Updated last year
• swj0419 / in-context-pretraining
  ☆49Updated last year
• NoviScl / GPT3-Reliability
  ☆78Updated 2 years ago
• GAIR-NLP / MetaCritique
  Evaluate the Quality of Critique
  ☆34Updated 11 months ago
• chang-github-00 / LLM-Predictive-Decoding
  ☆12Updated 5 months ago
• mega002 / ff-layers
  The accompanying code for "Transformer Feed-Forward Layers Are Key-Value Memories". Mor Geva, Roei Schuster, Jonathan Berant, and Omer Le…
  ☆91Updated 3 years ago
• FranxYao / FlanT5-CoT-Specialization
  Implementation of ICML 23 Paper: Specializing Smaller Language Models towards Multi-Step Reasoning.
  ☆130Updated last year
• Nanami18 / Snowballed_Hallucination
  ☆44Updated 8 months ago
• nkandpa2 / long_tail_knowledge
  Repo for the paper "Large Language Models Struggle to Learn Long-Tail Knowledge"
  ☆76Updated 2 years ago
• ruiqi-zhong / DescribeDistributionalDifferences
  Code for preprint: Summarizing Differences between Text Distributions with Natural Language
  ☆42Updated 2 years ago