Alternatives and similar repositories for gpqa:

Users that are interested in gpqa are comparing it to the libraries listed below

• OSU-NLP-Group / GrokkedTransformer
  Code for NeurIPS'24 paper 'Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization'
  ☆170Updated 2 weeks ago
• da03 / Internalize_CoT_Step_by_Step
  ☆115Updated 2 months ago
• felipemaiapolo / tinyBenchmarks
  Evaluating LLMs with fewer examples
  ☆139Updated 8 months ago
• allenai / WildBench
  Benchmarking LLMs with Challenging Tasks from Real Users
  ☆202Updated last month
• andyrdt / refusal_direction
  Code and results accompanying the paper "Refusal in Language Models Is Mediated by a Single Direction".
  ☆135Updated 2 months ago
• snap-stanford / MLAgentBench
  ☆256Updated 5 months ago
• Psycoy / MixEval
  The official evaluation suite and dynamic data release for MixEval.
  ☆230Updated last month
• ScalingIntelligence / Archon
  Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.
  ☆141Updated last month
• kanishkg / stream-of-search
  Repository for the paper Stream of Search: Learning to Search in Language
  ☆105Updated 4 months ago
• WildEval / ZeroEval
  A simple unified framework for evaluating LLMs
  ☆153Updated last month
• booydar / babilong
  BABILong is a benchmark for LLM evaluation using the needle-in-a-haystack approach.
  ☆167Updated 3 weeks ago
• meg-tong / sycophancy-eval
  datasets from the paper "Towards Understanding Sycophancy in Language Models"
  ☆64Updated last year
• EleutherAI / sae-auto-interp
  ☆124Updated this week
• veronica320 / Faithful-COT
  Code and data accompanying our paper on arXiv "Faithful Chain-of-Thought Reasoning".
  ☆157Updated 7 months ago
• allenai / reward-bench
  RewardBench: the first evaluation tool for reward models.
  ☆462Updated this week
• kaistAI / FLASK
  [ICLR 2024 Spotlight] FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets
  ☆212Updated 11 months ago
• callummcdougall / sae_vis
  Create feature-centric and prompt-centric visualizations for sparse autoencoders (like those from Anthropic's published research).
  ☆170Updated this week
• scicode-bench / SciCode
  A benchmark that challenges language models to code solutions for scientific problems
  ☆92Updated last week
• expz / quiet-star
  Implementation of the Quiet-STAR paper (https://arxiv.org/pdf/2403.09629.pdf)
  ☆44Updated 4 months ago
• lm-sys / llm-decontaminator
  Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"
  ☆294Updated 11 months ago
• steering-vectors / steering-vectors
  Steering vectors for transformer language models in Pytorch / Huggingface
  ☆69Updated 3 weeks ago
• LiveCodeBench / LiveCodeBench
  Official repository for the paper "LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code"
  ☆239Updated 2 months ago
• TransluceAI / observatory
  A toolkit for describing model features and intervening on those features to steer behavior.
  ☆132Updated last month
• princeton-nlp / USACO
  Can Language Models Solve Olympiad Programming?
  ☆104Updated 4 months ago
• EleutherAI / sae
  Sparse autoencoders
  ☆379Updated last week
• saprmarks / feature-circuits
  ☆115Updated 2 months ago
• StonyBrookNLP / appworld
  🌍 Repository for "AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agent", ACL'24 Best Resource Pap…
  ☆126Updated 2 weeks ago
• redwoodresearch / Easy-Transformer
  ☆104Updated 4 months ago
• justinchiu / openlogprobs
  Extract full next-token probabilities via language model APIs
  ☆230Updated 9 months ago