Alternatives and similar repositories for Tokenizer

Users that are interested in Tokenizer are comparing it to the libraries listed below

• TurkuNLP / Turku-neural-parser-pipeline
  A neural parsing pipeline for segmentation, morphological tagging, dependency parsing and lemmatization with pre-trained models for more …
  ☆114Updated last year
• mediacloud / sentence-splitter
  Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.
  ☆255Updated 2 years ago
• tsproisl / SoMaJo
  A tokenizer and sentence splitter for German and English web and social media texts.
  ☆147Updated 10 months ago
• MSeal / cython_hunspell
  Cython wrapper on Hunspell Dictionary
  ☆66Updated last year
• fnl / syntok
  Text tokenization and sentence segmentation (segtok v2)
  ☆206Updated 3 years ago
• Lilykos / pyphonetics
  A Python 3 phonetics library.
  ☆134Updated 5 years ago
• TakeLab / spacy-udpipe
  spaCy + UDPipe
  ☆163Updated 3 years ago
• dtuggener / CharSplit
  Compound splitter for German
  ☆108Updated 5 years ago
• sorenlind / lemmy
  🤘Lemmy is a lemmatizer for Danish 🇩🇰 and Swedish 🇸🇪
  ☆77Updated 4 years ago
• adbar / simplemma
  Simple multilingual lemmatizer for Python, especially useful for speed and efficiency
  ☆176Updated 4 months ago
• allo-media / text2num
  Parse and convert numbers written in French, English, Spanish, Portuguese, German and Catalan into their digit representation.
  ☆109Updated 4 months ago
• chrislit / abydos
  Abydos NLP/IR library for Python
  ☆191Updated 2 years ago
• nreimers / truecaser
  Language independent truecaser in Python.
  ☆160Updated 4 years ago
• proycon / python-frog
  Python bindings to the dutch NLP tool Frog (pos tagger, lemmatiser, NER tagger, morphological analysis, shallow parser, dependency parser…
  ☆49Updated 6 months ago
• clips / wordkit
  Featurize words into orthographic and phonological vectors.
  ☆41Updated 2 years ago
• riedlma / sequence_tagging
  Named Entity Recognition (LSTM + CRF + FastText) with models for [historic] German
  ☆26Updated 4 years ago
• aalto-speech / morfessor
  Morfessor is a tool for unsupervised and semi-supervised morphological segmentation
  ☆197Updated 5 years ago
• cadia-lvl / icelandic-NLP-resources
  Overview of Icelandic NLP resources at a glance
  ☆16Updated last year
• neocl / speach
  🐍🍑 Python 3 library for managing, annotating, and converting natural language corpuses using popular formats (CoNLL, ELAN, Praat, CSV, …
  ☆20Updated last year
• R1j1t / contextualSpellCheck
  ✔️Contextual word checker for better suggestions (not actively maintained)
  ☆417Updated 8 months ago
• yannvgn / laserembeddings
  LASER multilingual sentence embeddings as a pip package
  ☆224Updated 2 years ago
• fnl / segtok
  Segtok v2 is here: https://github.com/fnl/syntok -- A rule-based sentence segmenter (splitter) and a word tokenizer using orthographic fe…
  ☆170Updated 3 years ago
• JMendes1995 / py_heideltime
  ☆18Updated last year
• treble-ai / pyduckling
  Rust-based Python wrapper for duckling library in Haskell
  ☆25Updated 4 years ago
• globalwordnet / OMW
  The Open Multilingual Wordnet
  ☆64Updated last year
• pyconll / pyconll
  A minimal, pure Python library to interface with CoNLL-U format files.
  ☆152Updated this week
• tokestermw / spacy_hunspell
  Hunspell extension for spaCy 2.0.
  ☆94Updated last year
• bedapudi6788 / deepcorrect
  Text and Punctuation correction with Deep Learning
  ☆128Updated 5 years ago
• argilla-io / spacy-wordnet
  spacy-wordnet creates annotations that easily allow the use of wordnet and wordnet domains by using the nltk wordnet interface
  ☆260Updated last month
• aboSamoor / pycld2
  ☆174Updated 6 months ago