uhermjakob / utokenLinks
universal tokenizer
☆17Updated 3 years ago
Alternatives and similar repositories for utoken
Users that are interested in utoken are comparing it to the libraries listed below
Sorting:
- A python module for word inflections designed for use with spaCy.☆93Updated 5 years ago
- Curated corpus of parallel data derived from versions of the Bible provided by eBible.org.☆75Updated 4 months ago
- Python Finite-State Toolkit☆58Updated this week
- Bilingual sentence similarity classifier using Tensorflow☆24Updated 6 years ago
- Wiktra - Python tool of Wiktionary Transliteration modules for 514 languages and its 102 different scripts (orthographies)☆29Updated 2 months ago
- Master repo for the UniMorph project, includes the UniMorph schema and annotated data files☆32Updated 5 years ago
- Text tokenization and sentence segmentation (segtok v2)☆206Updated 3 years ago
- Scripts investigate, repair and normalize a wide range of text file problems at the character level.☆20Updated 2 years ago
- A modern, interlingual wordnet interface for Python☆260Updated 3 weeks ago
- MorphyNet: a Large Multilingual Database of Derivational and Inflectional Morphology (+morpheme segmentation)☆48Updated 2 years ago
- Transform TMX to text☆27Updated 2 years ago
- Efficient Low-Memory Aligner☆146Updated 8 months ago
- ☆75Updated last month
- A python true casing utility that restores case information for texts☆89Updated 2 years ago
- Python framework for processing Universal Dependencies data☆57Updated last week
- Various utilities for processing the data.☆212Updated last week
- This packages up data for the Open Multilingual Wordnet☆53Updated 3 months ago
- Python package for WikiMedia dump processing (Wiktionary, Wikipedia etc). Wikitext parsing, template expansion, Lua module execution. Fo…☆107Updated this week
- Efficient teacher-student models and scripts to make them☆51Updated last year
- OpusFilter - Parallel corpus processing toolkit☆110Updated last month
- OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.☆51Updated 2 months ago
- Runnable morphological analysis tools from the UniMorph project☆16Updated 6 years ago
- An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…☆25Updated 3 years ago
- Simple multilingual lemmatizer for Python, especially useful for speed and efficiency☆175Updated 3 months ago
- Character-level conversion between Hebrew text and Latin transliteration using deep learning - a demonstration of seq2seq training.☆14Updated 2 years ago
- Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.☆158Updated last year
- University of Colorado VerbNet☆113Updated last year
- downloads and parses subtitle dataset from opensubtitles.org☆16Updated last year
- bilingual dictionary extractor from parallel corpora☆22Updated 11 years ago
- UFSAC is a resource containing all WordNet Sense Annotated Corpora, and a Java library for manipulating them☆38Updated 3 years ago