wikimedia / sentencexLinks
A sentence segmentation library with wide language support optimized for speed and utility.
☆68Updated last week
Alternatives and similar repositories for sentencex
Users that are interested in sentencex are comparing it to the libraries listed below
Sorting:
- OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.☆52Updated 3 weeks ago
- Faster, modernized fork of the language identification tool langid.py☆59Updated 11 months ago
- Targetted language identifier, based on FastText and Hunspell.☆37Updated last month
- Multilingual sentence alignment using sentence embeddings☆127Updated 11 months ago
- Seed Machine Translation Data☆33Updated 11 months ago
- A modern, interlingual wordnet interface for Python☆267Updated last month
- Efficient teacher-student models and scripts to make them☆52Updated last year
- Multilingual syllable annotation pipeline component for spacy☆39Updated 2 years ago
- Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)☆75Updated 6 months ago
- Simple multilingual lemmatizer for Python, especially useful for speed and efficiency☆177Updated 4 months ago
- ☆78Updated 2 months ago
- Tool to fix bitexts and tag near-duplicates for removal☆33Updated last month
- Measure the readability of a given text using surface characteristics☆80Updated 9 months ago
- Library for fast text representation and classification.☆31Updated last year
- 80x faster and 95% accurate language identification with Fasttext☆161Updated last year
- Logical structure analysis for visually structured documents☆92Updated 3 years ago
- ☆31Updated last year
- OpusFilter - Parallel corpus processing toolkit☆110Updated 3 weeks ago
- 💥 Use Hugging Face text and token classification pipelines directly in spaCy☆63Updated last year
- Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.☆255Updated 2 years ago
- Python Finite-State Toolkit☆58Updated 2 weeks ago
- Searching in-memory corpus with Corpus Query Language (CQL)☆19Updated 10 months ago
- Text tokenization and sentence segmentation (segtok v2)☆206Updated 3 years ago
- A python module for word inflections designed for use with spaCy.☆93Updated 5 years ago
- 🕸 GlotWeb: Web Indexing for Low-Resource Languages -- under construction.☆15Updated 2 months ago
- Implementation of "SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages" paper, accepted to E…☆28Updated 2 years ago
- NTREX -- News Test References for MT Evaluation☆85Updated last year
- multimodal document analysis☆166Updated last year
- The Open Parallel Corpus☆76Updated 2 months ago
- Python3 bindings for the Compact Language Detector v3 (CLD3)☆154Updated 2 years ago