Alternatives and similar repositories for py3langid

Users that are interested in py3langid are comparing it to the libraries listed below

• wikimedia / sentencex
  A sentence segmentation library with wide language support optimized for speed and utility.
  ☆65Updated 9 months ago
• bitextor / warc2text
  Extracts plain text, language identification and more metadata from WARC records
  ☆22Updated 3 months ago
• mbanon / fastspell
  Targetted language identifier, based on FastText and Hunspell.
  ☆34Updated 3 months ago
• openlanguagedata / seed
  Seed Machine Translation Data
  ☆31Updated 6 months ago
• Helsinki-NLP / OPUS
  The Open Parallel Corpus
  ☆72Updated 2 months ago
• adbar / simplemma
  Simple multilingual lemmatizer for Python, especially useful for speed and efficiency
  ☆160Updated 2 weeks ago
• browsermt / students
  Efficient teacher-student models and scripts to make them
  ☆51Updated last year
• daltonfury42 / truecase
  A python true casing utility that restores case information for texts
  ☆88Updated 2 years ago
• LanguageMachines / ucto
  Unicode tokeniser. Ucto tokenizes text files: it separates words from punctuation, and splits sentences. It offers several other basic pr…
  ☆68Updated 3 months ago
• mediacloud / sentence-splitter
  Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.
  ☆246Updated 2 years ago
• Helsinki-NLP / OpusTools
  ☆72Updated 2 months ago
• aboSamoor / pycld2
  ☆171Updated 2 months ago
• tokestermw / spacy_grammar
  Language Tool style grammar handling with spaCy 2.0
  ☆42Updated 6 years ago
• laurieburchell / open-lid-dataset
  Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)
  ☆74Updated 2 months ago
• bjascob / pyInflect
  A python module for word inflections designed for use with spaCy.
  ☆92Updated 5 years ago
• bitextor / bicleaner
  Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.
  ☆157Updated 11 months ago
• zafercavdar / fasttext-langdetect
  80x faster and 95% accurate language identification with Fasttext
  ☆155Updated last year
• SYSTRAN / fuzzy-match
  Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.
  ☆50Updated last month
• dbmdz / deep-eos
  General-Purpose Neural Networks for Sentence Boundary Detection
  ☆73Updated 2 years ago
• tsproisl / SoMaJo
  A tokenizer and sentence splitter for German and English web and social media texts.
  ☆144Updated 5 months ago
• bitextor / bifixer
  Tool to fix bitexts and tag near-duplicates for removal
  ☆30Updated 3 months ago
• cldf / segments
  Unicode Standard tokenization routines and orthography profile segmentation
  ☆37Updated 3 months ago
• adbar / htmldate
  Fast and robust date extraction from web pages, with Python or on the command-line
  ☆127Updated 5 months ago
• explosion / spacy-lookups-data
  📂 Additional lookup tables and data resources for spaCy
  ☆105Updated 4 months ago
• doccano / doccano-client
  A simple client for doccano API.
  ☆85Updated last year
• AU-DIS / LSTM_langid
  Source code for the Apple reproduction
  ☆32Updated 4 years ago
• KennethEnevoldsen / spacy-wrap
  spaCy-wrap is a wrapper library for spaCy for including fine-tuned transformers from Huggingface in your spaCy pipeline allowing you to i…
  ☆46Updated last year
• allo-media / text2num
  Parse and convert numbers written in French, English, Spanish, Portuguese, German and Catalan into their digit representation.
  ☆107Updated last week
• Babelscape / wikineural
  Data and evaluation code for the paper WikiNEuRal: Combined Neural and Knowledge-based Silver Data Creation for Multilingual NER (EMNLP 2…
  ☆67Updated 2 years ago
• fnl / segtok
  Segtok v2 is here: https://github.com/fnl/syntok -- A rule-based sentence segmenter (splitter) and a word tokenizer using orthographic fe…
  ☆170Updated 3 years ago