Alternatives and similar repositories for Tokenizer

Users that are interested in Tokenizer are comparing it to the libraries listed below

• TurkuNLP / Turku-neural-parser-pipeline
  A neural parsing pipeline for segmentation, morphological tagging, dependency parsing and lemmatization with pre-trained models for more …
  ☆113Updated last year
• tokestermw / spacy_hunspell
  Hunspell extension for spaCy 2.0.
  ☆94Updated 10 months ago
• TakeLab / spacy-udpipe
  spaCy + UDPipe
  ☆161Updated 3 years ago
• pyconll / pyconll
  A minimal, pure Python library to interface with CoNLL-U format files.
  ☆151Updated 2 years ago
• nreimers / truecaser
  Language independent truecaser in Python.
  ☆160Updated 3 years ago
• Abhijit-2592 / spacy-langdetect
  A fully customisable language detection pipeline for spaCy
  ☆93Updated 6 years ago
• lunayach / visNER
  In the wild extraction of entities that are found using Flair and displayed using a very elegant front-end.
  ☆71Updated 2 years ago
• explosion / spacy-lookups-data
  📂 Additional lookup tables and data resources for spaCy
  ☆105Updated 3 weeks ago
• mpsilfve / finer-data
  ☆25Updated 5 years ago
• stephantul / piecelearn
  Learning BPE embeddings by first learning a segmentation model and then training word2vec
  ☆19Updated 2 years ago
• mideind / GreynirEngine
  A fast, efficient natural language processing engine for Icelandic.
  ☆62Updated 8 months ago
• mariananeves / annotation-tools
  ☆64Updated 2 years ago
• clips / wordkit
  Featurize words into orthographic and phonological vectors.
  ☆41Updated 2 years ago
• nert-nlp / streusle
  STREUSLE: a corpus with comprehensive lexical semantic annotation (multiword expressions, supersenses)
  ☆66Updated 3 weeks ago
• MartinoMensio / spacy-sentence-bert
  Sentence transformers models for SpaCy
  ☆107Updated 2 years ago
• tsproisl / SoMaJo
  A tokenizer and sentence splitter for German and English web and social media texts.
  ☆146Updated 6 months ago
• stefan-it / europeana-bert
  BERT and ELECTRA models trained on Europeana Newspapers
  ☆38Updated 3 years ago
• proycon / flat
  FoLiA Linguistic Annotation Tool -- Flat is a web-based linguistic annotation environment based around the FoLiA format (http://proycon.g…
  ☆112Updated 5 months ago
• dkpro / dkpro-cassis
  UIMA CAS processing library written in Python
  ☆90Updated last week
• Lilykos / pyphonetics
  A Python 3 phonetics library.
  ☆133Updated 5 years ago
• OpenJarbas / simple_NER
  simple rule based named entity recognition
  ☆42Updated 3 years ago
• paulrinckens / timexy
  A spaCy custom component that extracts and normalizes temporal expressions
  ☆54Updated 2 years ago
• MSeal / cython_hunspell
  Cython wrapper on Hunspell Dictionary
  ☆66Updated last year
• BramVanroy / spacy_conll
  Pipeline component for spaCy (and other spaCy-wrapped parsers such as spacy-stanza and spacy-udpipe) that adds CoNLL-U properties to a Do…
  ☆80Updated 11 months ago
• flairNLP / flair-lms
  Language Models for Zalando's flair library
  ☆61Updated 5 years ago
• dbmdz / deep-eos
  General-Purpose Neural Networks for Sentence Boundary Detection
  ☆73Updated 2 years ago
• cslu-nlp / DetectorMorse
  Fast supervised sentence boundary detection using the averaged perceptron
  ☆90Updated 6 years ago
• wjbmattingly / spacyex
  SpaCyEx allows the creation of spaCy Matcher patterns with RegEx like syntax.
  ☆59Updated last year
• hslh / magpie-corpus
  MAGPIE: A sense-annotated corpus of potentially idiomatic expressions
  ☆27Updated 5 years ago
• udapi / udapi-python
  Python framework for processing Universal Dependencies data
  ☆57Updated last week