adbar / py3langid
Faster, modernized fork of the language identification tool langid.py
☆55Updated 4 months ago
Alternatives and similar repositories for py3langid:
Users that are interested in py3langid are comparing it to the libraries listed below
- Targetted language identifier, based on FastText and Hunspell.☆34Updated 2 months ago
- A sentence segmentation library with wide language support optimized for speed and utility.☆61Updated 7 months ago
- Extracts plain text, language identification and more metadata from WARC records☆21Updated last month
- Fast and accurate natural language detection. Detector written in Python. Nito-ELD, ELD.☆17Updated last year
- Seed Machine Translation Data☆31Updated 5 months ago
- Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)☆72Updated 2 weeks ago
- Fast and robust date extraction from web pages, with Python or on the command-line☆124Updated 3 months ago
- A python true casing utility that restores case information for texts☆88Updated 2 years ago
- Boolean text search in Python☆45Updated 2 years ago
- A python module for word inflections designed for use with spaCy.☆92Updated 5 years ago
- Source code for the Apple reproduction☆32Updated 3 years ago
- Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.☆157Updated 10 months ago
- Python3 bindings for the Compact Language Detector v3 (CLD3)☆151Updated last year
- Simple multilingual lemmatizer for Python, especially useful for speed and efficiency☆154Updated 4 months ago
- Automatic extraction of edited sentences from text edition histories.☆83Updated 3 years ago
- General-Purpose Neural Networks for Sentence Boundary Detection☆73Updated 2 years ago
- Tool to fix bitexts and tag near-duplicates for removal☆30Updated 2 months ago
- Bilingual sentence similarity classifier using Tensorflow☆21Updated 5 years ago
- FoLiA Linguistic Annotation Tool -- Flat is a web-based linguistic annotation environment based around the FoLiA format (http://proycon.g…☆112Updated 2 months ago
- Generate a SQLite database from Wikipedia & Wikidata dumps.☆33Updated last year
- Parse and convert numbers written in French, English, Spanish, Portuguese, German and Catalan into their digit representation.☆106Updated 2 months ago
- A bidirectional recurrent neural network model with attention mechanism for restoring missing punctuation in unsegmented text☆36Updated 4 years ago
- Thot toolkit for statistical machine translation☆53Updated 2 years ago
- An asynchronous concurrent pipeline for classifying Common Crawl based on fastText's pipeline.☆86Updated 3 years ago
- Stand-alone WordNet API☆48Updated 3 years ago
- A simple client for doccano API.☆84Updated 10 months ago
- 80x faster and 95% accurate language identification with Fasttext☆152Updated last year
- SegEval Segmentation Evaluation Package☆56Updated last year
- an experimental implementation of Burrow's delta in Python 3☆21Updated 3 years ago
- Text tokenization and sentence segmentation (segtok v2)☆201Updated 3 years ago