Alternatives and similar repositories for tokenization-repair

Users that are interested in tokenization-repair are comparing it to the libraries listed below

• flexudy-pipe / sentence-doctor
  Many Natural Language Processing tasks rely on sentence boundary detection (SBD). Although amazing libraries like spacy provide state of …
  ☆61Updated 5 years ago
• facebookresearch / muss
  Code and models used in "MUSS Multilingual Unsupervised Sentence Simplification by Mining Paraphrases".
  ☆99Updated 2 years ago
• jacklxc / StandAloneSpellingCorrection
  Repository for Findings of EMNLP 2020 "Context-aware Stand-alone Neural Spelling Correction"
  ☆18Updated 4 years ago
• Helsinki-NLP / XED
  XED multilingual emotion datasets
  ☆64Updated 2 years ago
• lanwuwei / GigaBERT
  Zero-shot Transfer Learning from English to Arabic
  ☆30Updated 3 years ago
• mayhewsw / pytorch-truecaser
  A simple neural truecaser written in pytorch and allennlp.
  ☆33Updated last year
• writer / fitbert
  Use BERT to Fill in the Blanks
  ☆83Updated 3 years ago
• bitextor / bifixer
  Tool to fix bitexts and tag near-duplicates for removal
  ☆33Updated last month
• samueljamesbell / sequence-labeler
  Neural network sequence labeling model
  ☆11Updated 5 years ago
• nelson-liu / lexical-semantic-recognition
  ☆17Updated 2 years ago
• BramVanroy / spacy_conll
  Pipeline component for spaCy (and other spaCy-wrapped parsers such as spacy-stanza and spacy-udpipe) that adds CoNLL-U properties to a Do…
  ☆82Updated last year
• bojone / labse
  Language-agnostic BERT Sentence Embedding (LaBSE)
  ☆153Updated 5 years ago
• LightTag / sequence-labeling-with-transformers
  Examples for aligning, padding and batching sequence labeling data (NER) for use with pre-trained transformer models
  ☆64Updated 2 years ago
• shrutirij / ocr-post-correction
  ☆140Updated last year
• helboukkouri / character-bert-pretraining
  Code for pre-training CharacterBERT models (as well as BERT models).
  ☆34Updated 4 years ago
• mikahama / natas
  Python 3 library for processing historical English
  ☆67Updated last year
• stephantul / piecelearn
  Learning BPE embeddings by first learning a segmentation model and then training word2vec
  ☆19Updated 2 years ago
• TurkuNLP / wikibert
  BERT models for many languages created from Wikipedia texts
  ☆33Updated 5 years ago
• vgupta123 / P-SIF
  Source code for our AAAI 2020 paper P-SIF: Document Embeddings using Partition Averaging
  ☆35Updated 5 years ago
• ruathudo / post-ocr-correction
  ☆11Updated 3 years ago
• jhlau / acceptability_prediction
  Scripts and tools for doing unsupervised acceptability prediction.
  ☆14Updated 2 years ago
• UKPLab / acl2020-interactive-entity-linking
  ☆34Updated 2 years ago
• Helsinki-NLP / OpusFilter
  OpusFilter - Parallel corpus processing toolkit
  ☆110Updated last month
• mawentao277 / CharBERT
  CharBERT: Character-aware Pre-trained Language Model (COLING2020)
  ☆121Updated 4 years ago
• microsoft / LID-tool
  This code provides word level language identification tool for identifying language for individual words in Code-Mixed text. e.g. The tex…
  ☆55Updated 5 years ago
• plkumjorn / GrASP
  An implementation of GrASP (Shnarch et. al., 2017)
  ☆22Updated 3 years ago
• ntunlp / Zero-Shot-Cross-Lingual-NER
  A repository for our AAAI-2020 Cross-lingual-NER paper. Code will be updated shortly.
  ☆47Updated 2 years ago
• BramVanroy / astred
  An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…
  ☆25Updated 3 years ago
• daltonfury42 / truecase
  A python true casing utility that restores case information for texts
  ☆89Updated 2 years ago
• shijie-wu / neural-transducer
  This repo contains a set of neural transducer, e.g. sequence-to-sequence model, focusing on character-level tasks.
  ☆76Updated 2 years ago