Alternatives and similar repositories for Tokenizer:

Users that are interested in Tokenizer are comparing it to the libraries listed below

• MSeal / cython_hunspell
  Cython wrapper on Hunspell Dictionary
  ☆66Updated 6 months ago
• TakeLab / spacy-udpipe
  spaCy + UDPipe
  ☆161Updated 2 years ago
• udapi / udapi-python
  Python framework for processing Universal Dependencies data
  ☆55Updated 3 weeks ago
• cadia-lvl / icelandic-NLP-resources
  Overview of Icelandic NLP resources at a glance
  ☆16Updated 6 months ago
• allo-media / text2num
  Parse and convert numbers written in French, English, Spanish, Portuguese, German and Catalan into their digit representation.
  ☆104Updated 2 months ago
• luismond / tm2tb
  Bilingual term extractor
  ☆52Updated last year
• TurkuNLP / Turku-neural-parser-pipeline
  A neural parsing pipeline for segmentation, morphological tagging, dependency parsing and lemmatization with pre-trained models for more …
  ☆112Updated 8 months ago
• robertostling / eflomal
  Efficient Low-Memory Aligner
  ☆140Updated this week
• hiredscorelabs / seqtolang
  Multi-Langauge Identification
  ☆29Updated 5 months ago
• BramVanroy / astred
  An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…
  ☆22Updated 3 years ago
• yannvgn / laserembeddings
  LASER multilingual sentence embeddings as a pip package
  ☆224Updated last year
• fnl / syntok
  Text tokenization and sentence segmentation (segtok v2)
  ☆203Updated 2 years ago
• paulrinckens / timexy
  A spaCy custom component that extracts and normalizes temporal expressions
  ☆52Updated last year
• bjascob / pyInflect
  A python module for word inflections designed for use with spaCy.
  ☆92Updated 4 years ago
• Helsinki-NLP / OpusFilter
  OpusFilter - Parallel corpus processing toolkit
  ☆104Updated this week
• UCREL / pymusas
  Python Multilingual Ucrel Semantic Analysis System
  ☆31Updated 5 months ago
• pyconll / pyconll
  A minimal, pure Python library to interface with CoNLL-U format files.
  ☆149Updated last year
• jonfd / nefnir
  A lemmatizer for Icelandic text
  ☆16Updated 6 years ago
• bitextor / bicleaner
  Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.
  ☆152Updated 7 months ago
• nert-nlp / streusle
  STREUSLE: a corpus with comprehensive lexical semantic annotation (multiword expressions, supersenses)
  ☆64Updated last year
• Helsinki-NLP / OpusTools
  ☆68Updated 5 months ago
• steinst / SentAlign
  ☆25Updated last year
• rewicks / ersatz
  ☆44Updated 5 months ago
• tokestermw / spacy_hunspell
  Hunspell extension for spaCy 2.0.
  ☆94Updated 5 months ago
• writer / replaCy
  spaCy match and replace, maintaining conjugation
  ☆35Updated 2 years ago
• tylerneylon / explacy
  A small tool that EXPLains spACY parse results. See what I did there?
  ☆83Updated 2 years ago
• BramVanroy / spacy_conll
  Pipeline component for spaCy (and other spaCy-wrapped parsers such as spacy-stanza and spacy-udpipe) that adds CoNLL-U properties to a Do…
  ☆78Updated 6 months ago
• mariananeves / annotation-tools
  ☆64Updated last year
• liao961120 / concordancer
  Searching in-memory corpus with Corpus Query Language (CQL)
  ☆19Updated last month
• mediacloud / sentence-splitter
  Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.
  ☆234Updated 2 years ago