adbar / py3langidLinks
Faster, modernized fork of the language identification tool langid.py
☆60Updated last year
Alternatives and similar repositories for py3langid
Users that are interested in py3langid are comparing it to the libraries listed below
Sorting:
- Simple multilingual lemmatizer for Python, especially useful for speed and efficiency☆184Updated 8 months ago
- Python3 bindings for the Compact Language Detector v3 (CLD3)☆155Updated 2 years ago
- Targetted language identifier, based on FastText and Hunspell.☆38Updated 5 months ago
- A sentence segmentation library with wide language support optimized for speed and utility.☆86Updated 2 weeks ago
- 80x faster and 95% accurate language identification with Fasttext☆164Updated 2 years ago
- Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.☆255Updated 3 years ago
- Parse and convert numbers written in French, English, Spanish, Portuguese, German and Catalan into their digit representation.☆112Updated 3 weeks ago
- Fast and robust date extraction from web pages, with Python or on the command-line☆145Updated 3 months ago
- Seed Machine Translation Data☆33Updated last year
- The Open Parallel Corpus☆82Updated 3 weeks ago
- Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)☆74Updated 10 months ago
- ☆178Updated 10 months ago
- A modern, interlingual wordnet interface for Python☆280Updated this week
- Text tokenization and sentence segmentation (segtok v2)☆208Updated 3 years ago
- Searching in-memory corpus with Corpus Query Language (CQL)☆19Updated last year
- Translation demonstrator☆37Updated 5 years ago
- A tokenizer and sentence splitter for German and English web and social media texts.☆151Updated last year
- Cython wrapper on Hunspell Dictionary☆66Updated last year
- OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.☆57Updated 4 months ago
- Tool to fix bitexts and tag near-duplicates for removal☆34Updated 5 months ago
- Next-generation Punkt sentence boundary detection with zero dependencies☆27Updated 2 months ago
- 🕸 GlotWeb: Web Indexing for Low-Resource Languages -- under construction.☆17Updated 5 months ago
- A python true casing utility that restores case information for texts☆88Updated 3 years ago
- fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-ha…☆43Updated 3 years ago
- Boolean text search in Python☆46Updated 7 months ago
- Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.☆160Updated last year
- Translate HTML using Argos Translate☆57Updated 2 years ago
- Efficient teacher-student models and scripts to make them☆54Updated 2 years ago
- A versioned python wrapper package for cmudict (https://github.com/cmusphinx/cmudict).☆67Updated this week
- A Python library for working with and comparing language codes.☆353Updated 9 months ago