Alternatives and similar repositories for Tokenizer:

Users that are interested in Tokenizer are comparing it to the libraries listed below

• TakeLab / spacy-udpipe
  spaCy + UDPipe
  ☆161Updated 2 years ago
• tokestermw / spacy_hunspell
  Hunspell extension for spaCy 2.0.
  ☆94Updated 7 months ago
• writer / replaCy
  spaCy match and replace, maintaining conjugation
  ☆35Updated 2 years ago
• AU-DIS / LSTM_langid
  Source code for the Apple reproduction
  ☆32Updated 3 years ago
• tokestermw / spacy_kenlm
  KenLM extension for spaCy 2.0.
  ☆16Updated 7 years ago
• cadia-lvl / icelandic-NLP-resources
  Overview of Icelandic NLP resources at a glance
  ☆16Updated 9 months ago
• sloev / spacy-syllables
  Multilingual syllable annotation pipeline component for spacy
  ☆39Updated 2 years ago
• jonfd / nefnir
  A lemmatizer for Icelandic text
  ☆16Updated 7 years ago
• nreimers / truecaser
  Language independent truecaser in Python.
  ☆160Updated 3 years ago
• explosion / spacy-lookups-data
  📂 Additional lookup tables and data resources for spaCy
  ☆105Updated 2 months ago
• wjbmattingly / spacyex
  SpaCyEx allows the creation of spaCy Matcher patterns with RegEx like syntax.
  ☆59Updated 10 months ago
• thomasthiebaud / spacy-fastlang
  Language detection using Spacy and Fasttext
  ☆55Updated last year
• MartinoMensio / spacy-sentence-bert
  Sentence transformers models for SpaCy
  ☆107Updated 2 years ago
• paulrinckens / timexy
  A spaCy custom component that extracts and normalizes temporal expressions
  ☆54Updated 2 years ago
• tylerneylon / explacy
  A small tool that EXPLains spACY parse results. See what I did there?
  ☆83Updated 3 years ago
• pyconll / pyconll
  A minimal, pure Python library to interface with CoNLL-U format files.
  ☆149Updated last year
• tsproisl / SoMaJo
  A tokenizer and sentence splitter for German and English web and social media texts.
  ☆139Updated 3 months ago
• BramVanroy / astred
  An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…
  ☆22Updated 3 years ago
• adbar / simplemma
  Simple multilingual lemmatizer for Python, especially useful for speed and efficiency
  ☆154Updated 4 months ago
• Halvani / Constituent-Treelib
  A lightweight Python library for constructing, processing, and visualizing constituent trees.
  ☆67Updated 2 months ago
• clips / wordkit
  Featurize words into orthographic and phonological vectors.
  ☆40Updated last year
• cfournie / segmentation.evaluation
  SegEval Segmentation Evaluation Package
  ☆56Updated last year
• explosion / spacy-experimental
  🧪 Cutting-edge experimental spaCy components and features
  ☆98Updated 11 months ago
• Lilykos / pyphonetics
  A Python 3 phonetics library.
  ☆129Updated 5 years ago
• nickdavidhaynes / spacy-cld
  Language detection extension for spaCy 2.0+
  ☆112Updated 6 years ago
• Liebeck / spacy-iwnlp
  German lemmatization with IWNLP as extension for spaCy
  ☆24Updated last year
• BramVanroy / spacy_conll
  Pipeline component for spaCy (and other spaCy-wrapped parsers such as spacy-stanza and spacy-udpipe) that adds CoNLL-U properties to a Do…
  ☆80Updated 8 months ago
• bitextor / bifixer
  Tool to fix bitexts and tag near-duplicates for removal
  ☆30Updated last month
• kevin91nl / text-highlighter
  A Streamlit component for annotating text by text selecting.
  ☆40Updated 9 months ago
• UCREL / Multilingual-USAS
  Lexicons for the Multilingual UCREL Semantic Analysis System
  ☆41Updated last year