Alternatives and similar repositories for dedup

Users that are interested in dedup are comparing it to the libraries listed below

• ilinguistics / common_crawl_corpus
  Scripts for building a geo-located web corpus using Common Crawl data
  ☆11Updated 3 weeks ago
• transducens / linguacrawl
  Crawling engine that crawls a set of top-level domains looking for documents in a list of languages
  ☆11Updated 2 years ago
• xigt / freki
  Analyze XML extracted from PDFs (e.g. from TET or PDFMiner)
  ☆20Updated 8 years ago
• flexudy-pipe / sentence-doctor
  Many Natural Language Processing tasks rely on sentence boundary detection (SBD). Although amazing libraries like spacy provide state of …
  ☆62Updated 5 years ago
• DanielJDufour / date-extractor
  Extract dates from text
  ☆66Updated 5 years ago
• TurkuNLP / ocr-correction
  Post-processing OCR errors with seq2seq models
  ☆28Updated 5 years ago
• hiredscorelabs / seqtolang
  Multi-Langauge Identification
  ☆28Updated last year
• alexyorke / butter-fingers
  A python library to generate highly realistic typos (fuzz-testing)
  ☆13Updated 10 months ago
• Cheng-Lin-Li / KnowledgeGraph
  This repository for Web Crawling, Information Extraction, and Knowledge Graph build up.
  ☆33Updated 7 years ago
• kootenpv / textsearch
  Find strings/words in text; convenience and C speed
  ☆126Updated 3 years ago
• dataiku / dss-plugin-nlp-preparation
  Dataiku DSS plugin to detect languages, correct misspellings, and clean text data 🧼
  ☆22Updated 2 weeks ago
• JonathanRaiman / epub_conversion
  Python package for converting xml and epubs to text files
  ☆33Updated 5 years ago
• openlegaldata / legal-ner
  Named entity recognition for the legal domain
  ☆43Updated 4 years ago
• TyMick / loan-risk-neural-network
  Loan Risk Prediction Neural Network and API
  ☆17Updated 5 years ago
• anhaidgroup / py_stringsimjoin
  Scalable String Similarity Joins in Python
  ☆39Updated last year
• boberle / dependency2tree
  Convert CoNLL output of a dependency parser into a latex or graphviz tree
  ☆12Updated 5 years ago
• Flowinger / NLP-Reddit-Comments
  Using NLP to cluster reddit user comments by topics
  ☆14Updated 8 years ago
• argilla-io / biome-text
  Custom Natural Language Processing with big and small models 🌲🌱
  ☆66Updated 4 years ago
• vincent9514 / Text-Variant-Generation
  📄Neural Sentential Paraphrase Generation to Augment Chatbot Training Dataset
  ☆21Updated 3 years ago
• staeiou / arxiv_archive
  A tidy and complete archive of metadata for papers on arxiv.org, 1993-2019
  ☆29Updated 6 years ago
• ad-freiburg / tokenization-repair
  Correction of spaces with character-based neural language models.
  ☆13Updated 3 years ago
• MSeal / cython_hunspell
  Cython wrapper on Hunspell Dictionary
  ☆66Updated last year
• OpenJarbas / simple_NER
  simple rule based named entity recognition
  ☆42Updated 3 years ago
• tokestermw / spacy_kenlm
  KenLM extension for spaCy 2.0.
  ☆16Updated 8 years ago
• pmbaumgartner / spacy-setfit-textcat
  ☆30Updated 3 years ago
• domyounglee / TF-TrigramBlocking-transformer
  Transformer based Trigram Blocking implementation in Tensorflow
  ☆11Updated 5 years ago
• JarrodAJ / sec_employee_information_extraction
  NSS Capstone project to use natural language modeling, classification, and information extraction to get the exact employee count values …
  ☆16Updated 7 years ago
• Helsinki-NLP / sentimentator
  Tool for sentiment analysis annotation
  ☆13Updated 10 months ago
• mediacloud / date_guesser
  A library to extract a publication date from a web page, along with a measure of the accuracy.
  ☆41Updated 6 years ago
• litus-ai / classy
  classy is a simple-to-use library for building high-performance Machine Learning models in NLP.
  ☆87Updated last month