wikimedia / sentencexLinks
A sentence segmentation library with wide language support optimized for speed and utility.
☆86Updated 3 weeks ago
Alternatives and similar repositories for sentencex
Users that are interested in sentencex are comparing it to the libraries listed below
Sorting:
- Simple multilingual lemmatizer for Python, especially useful for speed and efficiency☆185Updated 8 months ago
- Faster, modernized fork of the language identification tool langid.py☆60Updated last year
- Seed Machine Translation Data☆33Updated last year
- A modern, interlingual wordnet interface for Python☆282Updated last week
- Extracts plain text, language identification and more metadata from WARC records☆23Updated 4 months ago
- The Open Parallel Corpus☆82Updated 3 weeks ago
- Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)☆74Updated 10 months ago
- A polite and user-friendly downloader for Common Crawl data☆67Updated 5 months ago
- OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.☆57Updated last week
- Multilingual sentence alignment using sentence embeddings☆139Updated last year
- ☆81Updated last week
- Aksharamukha Python Library☆56Updated last year
- Efficient teacher-student models and scripts to make them☆54Updated 2 years ago
- A python module for word inflections designed for use with spaCy.☆93Updated 6 years ago
- Targetted language identifier, based on FastText and Hunspell.☆38Updated 5 months ago
- A project about benchmarking and evaluating existing PDF extraction tools on their semantic abilities to extract the body texts from PDF …☆69Updated 5 years ago
- Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.☆255Updated 3 years ago
- Text tokenization and sentence segmentation (segtok v2)☆208Updated 3 years ago
- Next-generation Punkt sentence boundary detection with zero dependencies☆27Updated 2 months ago
- Library for fast text representation and classification.☆31Updated 2 years ago
- 80x faster and 95% accurate language identification with Fasttext☆164Updated 2 years ago
- an experimental implementation of Burrow's delta in Python 3☆21Updated 4 years ago
- ParaNames: A multilingual resource for parallel names☆39Updated last year
- Python3 bindings for the Compact Language Detector v3 (CLD3)☆155Updated 2 years ago
- Implementation of "SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages" paper, accepted to E…☆25Updated 3 years ago
- Multilingual syllable annotation pipeline component for spacy☆39Updated 2 years ago
- Cython wrapper on Hunspell Dictionary☆66Updated last year
- ☆34Updated 2 years ago
- 🔢 Work with static vector models☆36Updated 9 months ago
- Wiktra - Python tool of Wiktionary Transliteration modules for 514 languages and its 102 different scripts (orthographies)☆34Updated 7 months ago