Alternatives and similar repositories for Tokenizer:

Users that are interested in Tokenizer are comparing it to the libraries listed below

• cadia-lvl / icelandic-NLP-resources
  Overview of Icelandic NLP resources at a glance
  ☆16Updated 10 months ago
• BramVanroy / astred
  An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…
  ☆22Updated 3 years ago
• tokestermw / spacy_hunspell
  Hunspell extension for spaCy 2.0.
  ☆94Updated 8 months ago
• o-P-o / disagree
  Visualise, evaluate, and manage annotated data
  ☆33Updated 2 years ago
• nreimers / truecaser
  Language independent truecaser in Python.
  ☆160Updated 3 years ago
• cfournie / segmentation.evaluation
  SegEval Segmentation Evaluation Package
  ☆56Updated last year
• stephantul / piecelearn
  Learning BPE embeddings by first learning a segmentation model and then training word2vec
  ☆19Updated 2 years ago
• MSeal / cython_hunspell
  Cython wrapper on Hunspell Dictionary
  ☆66Updated 10 months ago
• tsproisl / SoMaJo
  A tokenizer and sentence splitter for German and English web and social media texts.
  ☆140Updated 4 months ago
• mideind / GreynirServer
  The greynir.is Icelandic natural language processing API and website.
  ☆65Updated 3 weeks ago
• clips / wordkit
  Featurize words into orthographic and phonological vectors.
  ☆40Updated last year
• NIHOPA / NLPre
  Python library for Natural Language Preprocessing (NLPre)
  ☆191Updated last year
• bootphon / pygamma-agreement
  Gamma Agreement in Python
  ☆43Updated last year
• jonfd / nefnir
  A lemmatizer for Icelandic text
  ☆16Updated 7 years ago
• dkpro / dkpro-cassis
  UIMA CAS processing library written in Python
  ☆88Updated last month
• wjbmattingly / spacyex
  SpaCyEx allows the creation of spaCy Matcher patterns with RegEx like syntax.
  ☆59Updated 11 months ago
• BramVanroy / spacy_conll
  Pipeline component for spaCy (and other spaCy-wrapped parsers such as spacy-stanza and spacy-udpipe) that adds CoNLL-U properties to a Do…
  ☆80Updated 9 months ago
• paulrinckens / timexy
  A spaCy custom component that extracts and normalizes temporal expressions
  ☆54Updated 2 years ago
• TakeLab / spacy-udpipe
  spaCy + UDPipe
  ☆161Updated 3 years ago
• lunayach / visNER
  In the wild extraction of entities that are found using Flair and displayed using a very elegant front-end.
  ☆71Updated 2 years ago
• doccano / doccano-client
  A simple client for doccano API.
  ☆85Updated 11 months ago
• MartinoMensio / spacy-sentence-bert
  Sentence transformers models for SpaCy
  ☆107Updated 2 years ago
• explosion / spacy-lookups-data
  📂 Additional lookup tables and data resources for spaCy
  ☆106Updated 3 months ago
• cdpierse / breame
  Lightweight utility tools for the detection of multiple spellings, meanings, and language-specific terminology in British and American En…
  ☆16Updated 3 years ago
• nickdavidhaynes / spacy-cld
  Language detection extension for spaCy 2.0+
  ☆112Updated 6 years ago
• mideind / GreynirEngine
  A fast, efficient natural language processing engine for Icelandic.
  ☆62Updated 6 months ago
• daltonfury42 / truecase
  A python true casing utility that restores case information for texts
  ☆88Updated 2 years ago
• jacksonllee / pylangacq
  Language Acquisition Research Tools
  ☆41Updated last year
• aboSamoor / pycld2
  ☆169Updated last month
• fnl / syntok
  Text tokenization and sentence segmentation (segtok v2)
  ☆201Updated 3 years ago