A unified interface for computing surprisal (log probabilities) from language models! Supports neural, symbolic, and black-box API models.
☆53Mar 20, 2026Updated 4 months ago
Alternatives and similar repositories for surprisal
Users that are interested in surprisal are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Tools for calculating psycholinguistically-relevant metrics of language statistics using transformer language models☆13Nov 11, 2022Updated 3 years ago
- SAP Benchmark☆29Sep 18, 2024Updated last year
- Simple tool for generating tokens with open source transformers and/or calculate per-token surprisal.☆14Jul 10, 2026Updated 2 weeks ago
- Utility for behavioral and representational analyses of Language Models☆193Jul 22, 2026Updated last week
- [Kauf & Ivanova, ACL 2023] A Better Way to Do Masked Language Model Scoring☆13Dec 1, 2023Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Code for "Multilingual language models predict human reading behavior"☆12Oct 9, 2022Updated 3 years ago
- Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization [ACL 2026]☆20Apr 18, 2026Updated 3 months ago
- Taxonomic Fragmentation of the NLI Task☆21Nov 11, 2020Updated 5 years ago
- A framework for nonlinear continuous-time regression☆41Jan 22, 2025Updated last year
- This repository is for the paper Metaphor Generation with Conceptual Mappings (ACL 2021).☆12Sep 24, 2021Updated 4 years ago
- Evaluate computational models on their alignment to behavioral and neural measurements in the domain of language☆42Jul 11, 2026Updated 2 weeks ago
- Pytorch implementation of next word prediction. Includes my own implementation of Google AI's Transformer architecture☆28Jun 30, 2021Updated 5 years ago
- Matlab and Python code for pupillometry studies.☆14May 27, 2021Updated 5 years ago
- A json Dataset with 25696 kpop songs. (2000 ~ 2023)☆11Dec 3, 2023Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- ☆10Nov 22, 2023Updated 2 years ago
- A corpus of short answers written by learners of English and graded with CEFR levels☆12Dec 17, 2021Updated 4 years ago
- Code for the paper "Implicit Representations of Meaning in Neural Language Models"☆57Feb 14, 2023Updated 3 years ago
- A neural language model that estimates incremental processing complexity☆40Oct 27, 2021Updated 4 years ago
- Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation☆15Aug 27, 2024Updated last year
- This project collects methods that enhance the comparison between AMR graphs.☆11Jun 15, 2023Updated 3 years ago
- Interactive visualization tool for multiple volumes, meshes and points based on VTK. The app can be controlled with Python scripts (optio…☆11Mar 13, 2022Updated 4 years ago
- Identifying the language of input text using character-level n-grams, with support for 45 languages☆11Dec 26, 2022Updated 3 years ago
- Official code for Conformal Isometry of Lie Group Representation in Recurrent Network of Grid Cells (NeurIPS workshop on Symmetry and Geo…☆13Nov 1, 2022Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- This repository contains the Potsdam Textbook Corpus (PoTeC) which is a natural reading eye-tracking corpus.☆16Jul 20, 2026Updated last week
- ☆16Mar 22, 2024Updated 2 years ago
- Logical inference system based on event semantics and degree semantics in formal semantics☆10Jan 22, 2023Updated 3 years ago
- ☆19Apr 22, 2026Updated 3 months ago
- ☆18Jan 3, 2025Updated last year
- A Python package for processing research with Minimalist grammars☆22Nov 13, 2021Updated 4 years ago
- thebeat: Rhythms in Python for science☆22Jul 20, 2026Updated last week
- Tool for the automatic assessment of lexical diversity☆14Sep 6, 2025Updated 10 months ago
- ☆20Jan 14, 2022Updated 4 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- quick and dirty dump of 25k English words from wordfreq☆16Oct 21, 2020Updated 5 years ago
- Alto, the Algebraic Language Toolkit☆18May 22, 2024Updated 2 years ago
- ☯️ AllenNLP training configurations for promising models on Named Entity Recognition. (BiLSTM-CRF, BiLSTM-CNN-CRF, BERT, BERT-CRF)☆15Nov 26, 2020Updated 5 years ago
- Code used for the "Envision Bootcamp" workshop☆14Sep 6, 2023Updated 2 years ago
- Implementation and experiments for Partially Supervised NER via Expected Entity Ratio in TACL 2022☆14Nov 7, 2022Updated 3 years ago
- A repo for LLM jailbreak☆14Sep 5, 2023Updated 2 years ago
- Encyclopedic Hub for Sentiment Dictionaries☆15Nov 20, 2025Updated 8 months ago