Alternatives and similar repositories for sentencex

Users that are interested in sentencex are comparing it to the libraries listed below

• laurieburchell / open-lid-dataset
  Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)
  ☆74Updated 2 months ago
• openlanguagedata / seed
  Seed Machine Translation Data
  ☆32Updated 7 months ago
• bitextor / warc2text
  Extracts plain text, language identification and more metadata from WARC records
  ☆22Updated 3 months ago
• hplt-project / OpusCleaner
  OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.
  ☆51Updated last week
• adbar / py3langid
  Faster, modernized fork of the language identification tool langid.py
  ☆56Updated 7 months ago
• danielvarga / hunalign
  Sentence aligner
  ☆114Updated 4 years ago
• Helsinki-NLP / OpusTools
  ☆74Updated 3 months ago
• bfsujason / bertalign
  Multilingual sentence alignment using sentence embeddings
  ☆120Updated 7 months ago
• sortiz / tmxt
  Transform TMX to text
  ☆27Updated 2 years ago
• bitextor / bicleaner
  Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.
  ☆157Updated last year
• liao961120 / concordancer
  Searching in-memory corpus with Corpus Query Language (CQL)
  ☆19Updated 6 months ago
• bltlab / paranames
  ParaNames: A multilingual resource for parallel names
  ☆34Updated last year
• bitextor / bifixer
  Tool to fix bitexts and tag near-duplicates for removal
  ☆30Updated 4 months ago
• mbanon / fastspell
  Targetted language identifier, based on FastText and Hunspell.
  ☆36Updated 4 months ago
• Helsinki-NLP / mammoth
  MAMMOTH: MAssively Multilingual Modular Open Translation @ Helsinki
  ☆23Updated 4 months ago
• isi-nlp / uroman
  Universal Romanizer that can convert any unicode script to roman (latin) script
  ☆210Updated 11 months ago
• jacksonllee / iso639
  ISO 639 language codes
  ☆45Updated 4 months ago
• Helsinki-NLP / OpusFilter
  OpusFilter - Parallel corpus processing toolkit
  ☆104Updated this week
• kpu / fasterText
  Library for fast text representation and classification.
  ☆30Updated last year
• thompsonb / vecalign
  Improved Sentence Alignment in Linear Time and Space
  ☆174Updated 2 years ago
• steinst / SentAlign
  ☆27Updated last year
• robertostling / eflomal
  Efficient Low-Memory Aligner
  ☆145Updated 5 months ago
• Helsinki-NLP / OPUS
  The Open Parallel Corpus
  ☆74Updated 3 months ago
• eole-nlp / eole
  Open language modeling toolkit based on PyTorch
  ☆129Updated last month
• WMT-QE-Task / wmt-qe-2023-data
  ☆9Updated last year
• adbar / simplemma
  Simple multilingual lemmatizer for Python, especially useful for speed and efficiency
  ☆164Updated 3 weeks ago
• cisnlp / Glot500
  Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023
  ☆103Updated last year
• explosion / spacy-huggingface-pipelines
  💥 Use Hugging Face text and token classification pipelines directly in spaCy
  ☆63Updated last year
• SYSTRAN / fuzzy-match
  Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.
  ☆50Updated 2 months ago
• alirezamshi / small100
  Implementation of "SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages" paper, accepted to E…
  ☆27Updated 2 years ago