benlipkin / probsem
Probabilistic LLM evaluations. [CogSci2023; ACL2023]
☆73Updated 8 months ago
Alternatives and similar repositories for probsem:
Users that are interested in probsem are comparing it to the libraries listed below
- The GitHub repo for Goal Driven Discovery of Distributional Differences via Language Descriptions☆69Updated 2 years ago
- Mechanistic Interpretability for Transformer Models☆50Updated 2 years ago
- Python library which enables complex compositions of language models such as scratchpads, chain of thought, tool use, selection-inference…☆206Updated 2 months ago
- ☆93Updated 3 months ago
- [Added T5 support to TRLX] A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)☆47Updated 2 years ago
- ☆34Updated last year
- ☆93Updated 10 months ago
- Code repository for the c-BTM paper☆106Updated last year
- Lightweight tools for quick and easy LLM demo's☆26Updated 6 months ago
- ☆44Updated 4 months ago
- ☆22Updated last year
- [ICML 2023] "Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation", Wenqing Zheng, S P Sharan, Ajay Kumar Jaiswal, …☆40Updated last year
- ☆89Updated last month
- One stop shop for all things carp☆59Updated 2 years ago
- [EMNLP'23] Execution-Based Evaluation for Open Domain Code Generation☆47Updated last year
- ☆73Updated 11 months ago
- A repository for transformer critique learning and generation☆89Updated last year
- Functional Benchmarks and the Reasoning Gap☆84Updated 6 months ago
- Factored Cognition Primer: How to write compositional language model programs☆48Updated 2 years ago
- Public Inflection Benchmarks☆68Updated last year
- CausalGym: Benchmarking causal interpretability methods on linguistic tasks☆41Updated 4 months ago
- Keeping language models honest by directly eliciting knowledge encoded in their activations.☆197Updated last week
- This repository contains code for cleaning your training data of benchmark data to help combat data snooping.☆25Updated last year
- A domain-specific probabilistic programming language for modeling and inference with language models☆119Updated last year
- Code and Data Repo for the CoNLL Paper -- Future Lens: Anticipating Subsequent Tokens from a Single Hidden State☆18Updated last year
- Experiments with generating opensource language model assistants☆97Updated last year
- A set of utilities for running few-shot prompting experiments on large-language models☆118Updated last year
- Multi-Domain Expert Learning☆67Updated last year
- TART: A plug-and-play Transformer module for task-agnostic reasoning☆195Updated last year
- A framework for pitting LLMs against each other in an evolving library of games ⚔☆32Updated this week