Alternatives and similar repositories for Tokenizer

Users that are interested in Tokenizer are comparing it to the libraries listed below

• cadia-lvl / icelandic-NLP-resources
  Overview of Icelandic NLP resources at a glance
  ☆16Updated 11 months ago
• mideind / GreynirEngine
  A fast, efficient natural language processing engine for Icelandic.
  ☆62Updated 8 months ago
• TakeLab / spacy-udpipe
  spaCy + UDPipe
  ☆161Updated 3 years ago
• Liebeck / spacy-iwnlp
  German lemmatization with IWNLP as extension for spaCy
  ☆24Updated last year
• nickdavidhaynes / spacy-cld
  Language detection extension for spaCy 2.0+
  ☆113Updated 6 years ago
• tokestermw / spacy_hunspell
  Hunspell extension for spaCy 2.0.
  ☆94Updated 10 months ago
• jonfd / nefnir
  A lemmatizer for Icelandic text
  ☆16Updated 7 years ago
• MSeal / cython_hunspell
  Cython wrapper on Hunspell Dictionary
  ☆66Updated 11 months ago
• TurkuNLP / Turku-neural-parser-pipeline
  A neural parsing pipeline for segmentation, morphological tagging, dependency parsing and lemmatization with pre-trained models for more …
  ☆113Updated last year
• unimorph / unimorph
  Master repo for the UniMorph project, includes the UniMorph schema and annotated data files
  ☆30Updated 5 years ago
• liao961120 / concordancer
  Searching in-memory corpus with Corpus Query Language (CQL)
  ☆19Updated 6 months ago
• nltk / nltk_contrib
  NLTK Contrib
  ☆166Updated last year
• paulrinckens / timexy
  A spaCy custom component that extracts and normalizes temporal expressions
  ☆54Updated 2 years ago
• ssharoff / biberpy
  Python version for Doug Biber's Multidimensional Analysis (MDA)
  ☆31Updated 3 months ago
• fnl / syntok
  Text tokenization and sentence segmentation (segtok v2)
  ☆205Updated 3 years ago
• MontrealCorpusTools / PolyglotDB
  Language data store and linguistic query API
  ☆40Updated last week
• flairNLP / flair-lms
  Language Models for Zalando's flair library
  ☆61Updated 5 years ago
• miotto / treetagger-python
  A Python module for interfacing with the Treetagger by Helmut Schmid.
  ☆75Updated this week
• fnl / segtok
  Segtok v2 is here: https://github.com/fnl/syntok -- A rule-based sentence segmenter (splitter) and a word tokenizer using orthographic fe…
  ☆170Updated 3 years ago
• sloev / spacy-syllables
  Multilingual syllable annotation pipeline component for spacy
  ☆39Updated 2 years ago
• MartinoMensio / spacy-sentence-bert
  Sentence transformers models for SpaCy
  ☆107Updated 2 years ago
• pyconll / pyconll
  A minimal, pure Python library to interface with CoNLL-U format files.
  ☆151Updated last year
• mariananeves / annotation-tools
  ☆64Updated 2 years ago
• udapi / udapi-python
  Python framework for processing Universal Dependencies data
  ☆57Updated last week
• kristopherkyle / corpus_toolkit
  A simple toolkit for conducting analyses using corpus methods
  ☆25Updated 3 years ago
• mpsilfve / finer-data
  ☆25Updated 5 years ago
• mediacloud / sentence-splitter
  Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.
  ☆249Updated 2 years ago
• robertostling / eflomal
  Efficient Low-Memory Aligner
  ☆144Updated 4 months ago
• explosion / spacy-lookups-data
  📂 Additional lookup tables and data resources for spaCy
  ☆105Updated this week
• mideind / GreynirServer
  The greynir.is Icelandic natural language processing API and website.
  ☆65Updated 2 weeks ago