This is the repository for MorphScore, a tokenizer evaluation framework for morphological alignment.
☆19Jul 10, 2025Updated last year
Alternatives and similar repositories for morphscore
Users that are interested in morphscore are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A tool to calculate dataset scaling needed to achieve cross-lingual training data equity.☆13Aug 14, 2024Updated 2 years ago
- Code for the paper "Greed is All You Need: An Evaluation of Tokenizer Inference Methods"☆15Sep 2, 2026Updated last month
- Simple-to-use scoring function for arbitrarily tokenized texts.☆52Oct 18, 2026Updated last week
- BPE modification that implements removing of the intermediate tokens during tokenizer training.☆27Nov 25, 2024Updated last year
- ☆11Jan 27, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆17Aug 22, 2026Updated 2 months ago
- Source code for paper Grammatical Error Correction in Low-Resource Scenarios (W-NUT 2019)☆13Jun 21, 2022Updated 4 years ago
- Metric Space Magnitude Computations☆15Jun 30, 2026Updated 4 months ago
- An initiative to collect and distribute resources for co-reference resolution in a unified standard.☆27May 12, 2024Updated 2 years ago
- Materials for "Natural Language Processing for Multilingual Task-Oriented Dialogue" Tutorial at ACL 2022☆14May 21, 2022Updated 4 years ago
- Code for SaGe subword tokenizer (EACL 2023)☆28Nov 30, 2024Updated last year
- AI SUGGEST is a powerful command-line assistant that leverages AI to provide accurate Linux commands based on natural language queries. S…☆11Aug 22, 2024Updated 2 years ago
- Code for "Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding" (EMNLP 2020).☆11May 1, 2025Updated last year
- ☆27Aug 19, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Type-and-example directed program synthesis using example propagation, as described in Program Synthesis Using Example Propagation.☆16Jul 26, 2026Updated 3 months ago
- Official repository for the EMNLP 2019 paper, "Learning Invariant Representations of Social Media Users."☆12Aug 27, 2021Updated 5 years ago
- 儿童故事常识推理与寓意理解评测（Commonsense Reasoning and Moral Understanding Evaluation in Children's Stories，CRMU）☆18Oct 22, 2024Updated 2 years ago
- [ACL 2025] TokAlign: Efficient Vocabulary Adaptation via Token Alignment☆12Jun 4, 2025Updated last year
- ☆14Feb 1, 2024Updated 2 years ago
- ☆12Apr 24, 2025Updated last year
- A python library for easily querying morphological inflection models trained on Unimorph☆13Oct 23, 2022Updated 4 years ago
- Research code for the paper "How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models"☆27Oct 3, 2021Updated 5 years ago
- Fonts code (http://ilab.usc.edu/datasets/fonts)☆11Apr 14, 2021Updated 5 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- SIGIR 2023 tutorial on cross language information retrieval.☆13Feb 28, 2024Updated 2 years ago
- ☆15Aug 2, 2024Updated 2 years ago
- A benchmark with locally sourced multilingual questions for 31 languages.☆18Oct 5, 2026Updated 3 weeks ago
- Calculates bounds on the sofa moving problem☆14Sep 12, 2019Updated 7 years ago
- Code for training CamemBERTa, a French language model based on DeBERTa V3. Development is done here https://gitlab.inria.fr/almanach/Came…☆13Nov 18, 2024Updated last year
- NeuralType (.ntf): a post-OpenType font format where the font is a small neural model☆14Oct 8, 2026Updated 3 weeks ago
- Code for the paper "BPE stays on SCRIPT", "Which Pieces Does Unigram Tokenization Really Need?" and MinGram☆22Oct 21, 2026Updated last week
- A Multilingual Keyboard Layout-Based Typo Generator☆18Nov 23, 2025Updated 11 months ago
- Can Large Language Models Identify Authorship? (EMNLP 2024 Findings)☆14Feb 4, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ML Reproducibility Challenge 2020: Electra reimplementation using PyTorch and Transformers☆11Apr 16, 2021Updated 5 years ago
- Basic Arabic shaping for RoboFont 3.4☆10Jan 12, 2026Updated 9 months ago
- ☆16Oct 6, 2026Updated 3 weeks ago
- ACL 2021 paper "Style is NOT a single variable: Case Studies for Cross-Style Language Understanding " by Dongyeop Kang and Eduard Hovy☆15Jul 19, 2021Updated 5 years ago
- Tools for calculating psycholinguistically-relevant metrics of language statistics using transformer language models☆13Nov 11, 2022Updated 3 years ago
- PhD thesis template with title page according to the University of Amsterdam.☆17Sep 12, 2021Updated 5 years ago
- ☆13Jul 10, 2020Updated 6 years ago