currentslab / fastlangidLinks
fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-hant)
☆43Updated 3 years ago
Alternatives and similar repositories for fastlangid
Users that are interested in fastlangid are comparing it to the libraries listed below
Sorting:
- A python true casing utility that restores case information for texts☆88Updated 3 years ago
- An asynchronous concurrent pipeline for classifying Common Crawl based on fastText's pipeline.☆86Updated 4 years ago
- A simple neural truecaser written in pytorch and allennlp.☆33Updated last year
- ☆178Updated 10 months ago
- A python module for word inflections designed for use with spaCy.☆93Updated 6 years ago
- Python3 bindings for the Compact Language Detector v3 (CLD3)☆155Updated 2 years ago
- Tool to fix bitexts and tag near-duplicates for removal☆34Updated 5 months ago
- Examples for aligning, padding and batching sequence labeling data (NER) for use with pre-trained transformer models☆64Updated 3 years ago
- Sentence transformers models for SpaCy☆108Updated 2 years ago
- Automatic extraction of edited sentences from text edition histories.☆83Updated 3 years ago
- BERT models for many languages created from Wikipedia texts☆33Updated 5 years ago
- ☆30Updated 3 years ago
- Implementation of Z-BERT-A: a zero-shot pipeline for unknown intent detection.☆44Updated 2 years ago
- Lightning Fast Language Prediction 🚀☆167Updated 5 months ago
- ☆32Updated 4 years ago
- SImple SenTence EmbeddeR☆74Updated 2 years ago
- Use BERT to Fill in the Blanks☆84Updated 4 years ago
- Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.☆255Updated 3 years ago
- zero-vocab or low-vocab embeddings☆18Updated 3 years ago
- Seed Machine Translation Data☆33Updated last year
- Corpus preprocessing☆99Updated last year
- A lightweight but powerful library to build token indices for NLP tasks, compatible with major Deep Learning frameworks like PyTorch and …☆51Updated last year
- 🤖📇 handling multiple nlp task in one pipeline☆57Updated 4 months ago
- OpenNeuroSpell contains parts of NeuroSpell (http://neurospell.com/en.php) released as open-source. More code will be published as soon a…☆20Updated last year
- General-Purpose Neural Networks for Sentence Boundary Detection☆73Updated 2 years ago
- Bilingual sentence similarity classifier using Tensorflow☆24Updated 6 years ago
- Crawling engine that crawls a set of top-level domains looking for documents in a list of languages☆11Updated 2 years ago
- Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation☆15Updated last year
- ICU based universal language tokenizer☆33Updated 4 years ago
- Tools for extracting parallel corpora from article titles across languages in Wikipedia☆74Updated 10 years ago