currentslab / fastlangid
fastlangid, the only language identification package that support cantonese (zh-yue), simplified (zh-hans) and traditional chinese (zh-hant)
☆39Updated 2 years ago
Alternatives and similar repositories for fastlangid:
Users that are interested in fastlangid are comparing it to the libraries listed below
- Tool to fix bitexts and tag near-duplicates for removal☆30Updated 2 months ago
- BERT models for many languages created from Wikipedia texts☆33Updated 4 years ago
- A simple neural truecaser written in pytorch and allennlp.☆33Updated 10 months ago
- Tower Parse: Low-Resource Dependency Parsing via Hierarchical Source Selection☆15Updated 3 years ago
- A web interface to understand language-specific BERT-models☆17Updated last year
- ☆33Updated 3 years ago
- GrammarTagger — A Neural Multilingual Grammar Profiler for Language Learning☆27Updated 4 years ago
- A python true casing utility that restores case information for texts☆88Updated 2 years ago
- A tiny BERT for low-resource monolingual models☆31Updated 6 months ago
- An Interactive Tool for Annotating Discourse Structure and Text Improvement☆16Updated 3 years ago
- Examples for aligning, padding and batching sequence labeling data (NER) for use with pre-trained transformer models☆65Updated 2 years ago
- An asynchronous concurrent pipeline for classifying Common Crawl based on fastText's pipeline.☆86Updated 4 years ago
- A library for data streaming and augmentation☆20Updated last year
- Bicleaner fork that uses neural networks☆40Updated 8 months ago
- This repository contains the code for the paper 'PARM: Paragraph Aggregation Retrieval Model for Dense Document-to-Document Retrieval' pu…☆40Updated 3 years ago
- Implementation of Nested Named Entity Recognition using Flair☆24Updated 3 years ago
- Repository with code for MaChAmp: https://aclanthology.org/2021.eacl-demos.22/☆86Updated last week
- 💫 A spaCy package for Yohei Tamura's Rust tokenizations library☆29Updated last year
- Code and datasets of "Multilingual Extractive Reading Comprehension by Runtime Machine Translation"☆40Updated 6 years ago
- Learning BPE embeddings by first learning a segmentation model and then training word2vec☆19Updated 2 years ago
- Bilingual sentence similarity classifier using Tensorflow☆21Updated 5 years ago
- Generate BERT vocabularies and pretraining examples from Wikipedias☆18Updated 4 years ago
- Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translation☆14Updated 7 months ago
- ☆30Updated 2 years ago
- Summary Explorer is a tool to visually explore the state-of-the-art in text summarization.☆44Updated 11 months ago
- python package for unsupervised text segmentation.☆14Updated 8 years ago
- List of corpora annotated for coreference for different languages☆17Updated 8 months ago
- Tool for parsing and converting various span encoding schemes.☆23Updated last year
- ☆169Updated 3 weeks ago
- Project OCELoT: an Open, Collaborative Evaluation Leaderboard of Translations☆20Updated 9 months ago