Alternatives and similar repositories for Tokenizer:

Users that are interested in Tokenizer are comparing it to the libraries listed below

• BramVanroy / astred
  An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…
  ☆22Updated 3 years ago
• MSeal / cython_hunspell
  Cython wrapper on Hunspell Dictionary
  ☆66Updated 7 months ago
• explosion / spacy-experimental
  🧪 Cutting-edge experimental spaCy components and features
  ☆96Updated 9 months ago
• globalwordnet / OMW
  The Open Multilingual Wordnet
  ☆61Updated 9 months ago
• ljos / navnkjenner
  Named-Entity Recognition for Norwegian Bokmål and Nynorsk
  ☆12Updated 5 years ago
• BramVanroy / spacy_conll
  Pipeline component for spaCy (and other spaCy-wrapped parsers such as spacy-stanza and spacy-udpipe) that adds CoNLL-U properties to a Do…
  ☆79Updated 7 months ago
• udapi / udapi-python
  Python framework for processing Universal Dependencies data
  ☆55Updated last week
• dkpro / dkpro-cassis
  UIMA CAS processing library written in Python
  ☆86Updated 9 months ago
• tsproisl / textcomplexity
  Linguistic and stylistic complexity measures for (literary) texts
  ☆79Updated last year
• omwn / omw-data
  This packages up data for the Open Multilingual Wordnet
  ☆45Updated last week
• lunayach / visNER
  In the wild extraction of entities that are found using Flair and displayed using a very elegant front-end.
  ☆71Updated 2 years ago
• luismsgomes / mosestokenizer
  ☆20Updated 3 years ago
• TakeLab / spacy-udpipe
  spaCy + UDPipe
  ☆160Updated 2 years ago
• tsproisl / SoMaJo
  A tokenizer and sentence splitter for German and English web and social media texts.
  ☆138Updated 2 months ago
• boberle / dependency2tree
  Convert CoNLL output of a dependency parser into a latex or graphviz tree
  ☆12Updated 4 years ago
• cadia-lvl / icelandic-NLP-resources
  Overview of Icelandic NLP resources at a glance
  ☆16Updated 8 months ago
• microsoft / spacy-ann-linker
  spaCy pipeline component for generating spaCy KnowledgeBase Alias Candidates for Entity Linking
  ☆85Updated 2 years ago
• bitextor / bifixer
  Tool to fix bitexts and tag near-duplicates for removal
  ☆29Updated 2 weeks ago
• paulrinckens / timexy
  A spaCy custom component that extracts and normalizes temporal expressions
  ☆54Updated 2 years ago
• fnl / syntok
  Text tokenization and sentence segmentation (segtok v2)
  ☆201Updated 2 years ago
• GorkaUrbizu / Coreference-Corpora-Resources
  List of corpora annotated for coreference for different languages
  ☆17Updated 6 months ago
• tsproisl / SoMeWeTa
  A part-of-speech tagger with support for domain adaptation and external resources.
  ☆22Updated 2 years ago
• MartinoMensio / spacy-sentence-bert
  Sentence transformers models for SpaCy
  ☆107Updated last year
• pyconll / pyconll
  A minimal, pure Python library to interface with CoNLL-U format files.
  ☆148Updated last year
• OnlpLab / NEMO-Corpus
  Named Entity (NER) annotations of the Hebrew Treebank (Haaretz newspaper) corpus, including: morpheme and token level NER labels, nested …
  ☆10Updated 3 years ago
• tokestermw / spacy_hunspell
  Hunspell extension for spaCy 2.0.
  ☆94Updated 6 months ago
• TurkuNLP / Turku-neural-parser-pipeline
  A neural parsing pipeline for segmentation, morphological tagging, dependency parsing and lemmatization with pre-trained models for more …
  ☆112Updated 9 months ago
• tylerneylon / explacy
  A small tool that EXPLains spACY parse results. See what I did there?
  ☆83Updated 2 years ago
• mideind / GreynirServer
  The greynir.is Icelandic natural language processing API and website.
  ☆65Updated 6 months ago
• Riccorl / transformer-srl
  Reimplementation of a BERT based model (Shi et al, 2019), currently the state-of-the-art for English SRL. This model implements also pred…
  ☆70Updated 2 years ago