adbar / py3langid
Faster, modernized fork of the language identification tool langid.py
☆53Updated 2 months ago
Alternatives and similar repositories for py3langid:
Users that are interested in py3langid are comparing it to the libraries listed below
- Targetted language identifier, based on FastText and Hunspell.☆34Updated last week
- Fast and accurate natural language detection. Detector written in Python. Nito-ELD, ELD.☆15Updated last year
- Seed Machine Translation Data☆30Updated 3 months ago
- A sentence segmentation library with wide language support optimized for speed and utility.☆58Updated 5 months ago
- Simple multilingual lemmatizer for Python, especially useful for speed and efficiency☆151Updated 3 months ago
- Source code for the Apple reproduction☆31Updated 3 years ago
- 💥 Use Hugging Face text and token classification pipelines directly in spaCy☆63Updated 11 months ago
- Python3 bindings for the Compact Language Detector v3 (CLD3)☆150Updated last year
- fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-ha…☆39Updated 2 years ago
- A python module for word inflections designed for use with spaCy.☆92Updated 5 years ago
- 80x faster and 95% accurate language identification with Fasttext☆146Updated last year
- Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.☆154Updated 8 months ago
- Parse and convert numbers written in French, English, Spanish, Portuguese, German and Catalan into their digit representation.☆105Updated last week
- Library for fast text representation and classification.☆28Updated last year
- Python Finite-State Toolkit☆50Updated last month
- Tool to fix bitexts and tag near-duplicates for removal☆29Updated 2 weeks ago
- Boolean text search in Python☆45Updated 2 years ago
- Extracts plain text, language identification and more metadata from WARC records☆21Updated 2 weeks ago
- Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)☆70Updated 9 months ago
- Cython wrapper on Hunspell Dictionary☆66Updated 7 months ago
- 🧪 Cutting-edge experimental spaCy components and features☆96Updated 9 months ago
- A simple client for doccano API.☆84Updated 8 months ago
- A python true casing utility that restores case information for texts☆88Updated 2 years ago
- Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.☆47Updated last month
- Fast and robust date extraction from web pages, with Python or on the command-line☆122Updated last month
- spaCy-wrap is a wrapper library for spaCy for including fine-tuned transformers from Huggingface in your spaCy pipeline allowing you to i…☆46Updated 10 months ago
- Text tokenization and sentence segmentation (segtok v2)☆201Updated 2 years ago
- A robust web archive analytics toolkit☆99Updated 2 months ago
- Deutsches Lyrik Korpus (DLK) / German Poetry Corpus☆18Updated 9 months ago
- General-Purpose Neural Networks for Sentence Boundary Detection☆72Updated last year