uhermjakob / utokenLinks
universal tokenizer
☆16Updated 3 years ago
Alternatives and similar repositories for utoken
Users that are interested in utoken are comparing it to the libraries listed below
Sorting:
- Bilingual sentence similarity classifier using Tensorflow☆24Updated 6 years ago
- A python module for word inflections designed for use with spaCy.☆93Updated 5 years ago
- Efficient Low-Memory Aligner☆146Updated 10 months ago
- MorphyNet: a Large Multilingual Database of Derivational and Inflectional Morphology (+morpheme segmentation)☆52Updated 2 years ago
- Efficient teacher-student models and scripts to make them☆52Updated last year
- Curated corpus of parallel data derived from versions of the Bible provided by eBible.org.☆76Updated 6 months ago
- A modern, interlingual wordnet interface for Python☆274Updated this week
- Scripts investigate, repair and normalize a wide range of text file problems at the character level.☆20Updated 3 years ago
- OpusFilter - Parallel corpus processing toolkit☆112Updated 2 weeks ago
- A list of vocabulary lists☆22Updated 5 years ago
- A set of pipelines for performing experiments on various NLP tasks with a focus on resource-poor/minority languages.☆36Updated last week
- Wiktra - Python tool of Wiktionary Transliteration modules for 514 languages and its 102 different scripts (orthographies)☆31Updated 5 months ago
- ☆78Updated 3 months ago
- Python framework for processing Universal Dependencies data☆57Updated last week
- Transform TMX to text☆28Updated 3 years ago
- Python Finite-State Toolkit☆60Updated last week
- Translation Memory Open-source Purifier☆35Updated 3 years ago
- Simple multilingual lemmatizer for Python, especially useful for speed and efficiency☆179Updated 5 months ago
- An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…☆25Updated 4 years ago
- Runnable morphological analysis tools from the UniMorph project☆16Updated 7 years ago
- Python package for WikiMedia dump processing (Wiktionary, Wikipedia etc). Wikitext parsing, template expansion, Lua module execution. Fo…☆108Updated this week
- This packages up data for the Open Multilingual Wordnet☆56Updated 5 months ago
- Put together a multilingual corpus from a variety of sources. Used for wordfreq and word embeddings.☆56Updated 4 years ago
- An NLP pipeline for Hebrew☆40Updated 5 months ago
- Sentence aligner☆121Updated 4 years ago
- This Python module can be used to obtain antonyms, synonyms, hypernyms, hyponyms, homophones and definitions.☆125Updated last year
- OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.☆52Updated last month
- Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.☆160Updated last year
- A tool that locates, downloads, and extracts machine translation corpora☆159Updated 2 months ago
- Improved Sentence Alignment in Linear Time and Space☆185Updated 2 years ago