adbar / py3langid
Faster, modernized fork of the language identification tool langid.py
☆48Updated 5 months ago
Related projects ⓘ
Alternatives and complementary repositories for py3langid
- Targetted language identifier, based on FastText and Hunspell.☆29Updated last month
- Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)☆67Updated 6 months ago
- Extracts plain text, language identification and more metadata from WARC records☆20Updated 3 months ago
- Source code for the Apple reproduction☆31Updated 3 years ago
- Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.☆150Updated 5 months ago
- A python module for word inflections designed for use with spaCy.☆92Updated 4 years ago
- In the wild extraction of entities that are found using Flair and displayed using a very elegant front-end.☆69Updated last year
- Tool to fix bitexts and tag near-duplicates for removal☆29Updated 3 months ago
- Corpus preprocessing☆95Updated 8 months ago
- ☆67Updated 3 months ago
- 💥 Use Hugging Face text and token classification pipelines directly in spaCy☆62Updated 8 months ago
- BERT and ELECTRA models trained on Europeana Newspapers☆36Updated 2 years ago
- A tiny BERT for low-resource monolingual models☆29Updated last month
- Seed Machine Translation Data☆30Updated last week
- Searching in-memory corpus with Corpus Query Language (CQL)☆18Updated 3 years ago
- OpusFilter - Parallel corpus processing toolkit☆102Updated 3 months ago
- Automatic extraction of edited sentences from text edition histories.☆81Updated 2 years ago
- A python true casing utility that restores case information for texts☆87Updated 2 years ago
- Transform TMX to text☆29Updated last year
- An easy-to-use library to linguistically compare one sentence and its words to another, in the same language or a different one. For inst…☆21Updated 2 years ago
- Python Finite-State Toolkit☆45Updated last week
- 🧪 Cutting-edge experimental spaCy components and features☆95Updated 6 months ago
- Bilingual sentence similarity classifier using Tensorflow☆19Updated 5 years ago
- Bicleaner fork that uses neural networks☆38Updated 3 months ago
- Identifying Historical People, Places and other Entities: Shared Task on Named Entity Recognition and Linking on Historical Newspapers at…☆22Updated 3 months ago
- A database of number names for 186 languages, locales, and scripts☆66Updated last year
- Parse and convert numbers written in French, English, Spanish, Portuguese, German and Catalan into their digit representation.☆103Updated 2 weeks ago
- 80x faster and 95% accurate language identification with Fasttext☆141Updated 9 months ago
- Python 3 library for processing historical English☆64Updated 3 months ago
- Translation demonstrator☆27Updated 4 years ago