Alternatives and similar repositories for utoken:

Users that are interested in utoken are comparing it to the libraries listed below

• SYSTRAN / similarity
  Bilingual sentence similarity classifier using Tensorflow
  ☆19Updated 5 years ago
• MeLeLBGU / SaGe
  Code for SaGe subword tokenizer (EACL 2023)
  ☆22Updated last month
• CUNY-CL / citylex
  An English lexical database from the Big 🍎, let's go Mets baby love da Mets
  ☆14Updated 2 months ago
• rali-udem / gophi
  GOPHI: an AMR-to-English Verbalizer
  ☆11Updated 4 years ago
• browsermt / students
  Efficient teacher-student models and scripts to make them
  ☆49Updated last year
• sortiz / tmxt
  Transform TMX to text
  ☆29Updated 2 years ago
• BramVanroy / astred
  An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…
  ☆22Updated 3 years ago
• zseder / hundict
  bilingual dictionary extractor from parallel corpora
  ☆22Updated 10 years ago
• delph-in / erg
  English Resource Grammar
  ☆20Updated 5 months ago
• udapi / udapi-python
  Python framework for processing Universal Dependencies data
  ☆56Updated 3 weeks ago
• mhulden / pyfoma
  Python Finite-State Toolkit
  ☆47Updated last week
• SYSTRAN / fuzzy-match
  Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.
  ☆46Updated 3 weeks ago
• akb89 / witokit
  A Python toolkit to generate a tokenized dump of Wikipedia for NLP
  ☆11Updated 8 months ago
• bjascob / pyInflect
  A python module for word inflections designed for use with spaCy.
  ☆92Updated 4 years ago
• snukky / wikiedits
  Automatic extraction of edited sentences from text edition histories.
  ☆82Updated 2 years ago
• cisnlp / GlotScript
  Resource and Tool for Writing System Identification -- LREC 2024
  ☆13Updated 7 months ago
• lgessler / microbert
  A tiny BERT for low-resource monolingual models
  ☆31Updated 3 months ago
• AU-DIS / LSTM_langid
  Source code for the Apple reproduction
  ☆31Updated 3 years ago
• Helsinki-NLP / OPUS-translator
  Translation demonstrator
  ☆29Updated 4 years ago
• explosion / spacy-alignments
  💫 A spaCy package for Yohei Tamura's Rust tokenizations library
  ☆27Updated last year
• neocl / speach
  🐍🍑 Python 3 library for managing, annotating, and converting natural language corpuses using popular formats (CoNLL, ELAN, Praat, CSV, …
  ☆17Updated 6 months ago
• jonathandunn / corpus_similarity
  Measure the similarity of text corpora for 74 languages
  ☆13Updated 11 months ago
• bltlab / mot
  Multilingual Open Text
  ☆25Updated 2 months ago
• bitextor / warc2text
  Extracts plain text, language identification and more metadata from WARC records
  ☆20Updated 5 months ago
• SemiringInc / Mueller-Report-Corpus
  The Mueller Report Corpus V 0.1
  ☆11Updated 4 years ago
• SapienzaNLP / xl-amr
  XL-AMR is a sequence-to-graph cross-lingual AMR parser that exploits transfer learning (EMNLP2020).
  ☆16Updated 5 months ago
• zaemyung / sentsplit
  A flexible sentence segmentation library using CRF model and regex rules
  ☆28Updated 10 months ago
• huggingface / gaia
  Hugging Face and Pyserini interoperability
  ☆20Updated last year
• hslh / magpie-corpus
  MAGPIE: A sense-annotated corpus of potentially idiomatic expressions
  ☆26Updated 4 years ago
• bjascob / amr_coref
  A python library / model for creating co-references between AMR graph nodes.
  ☆9Updated 2 years ago