ehsanasgari / 1000Langs
Creating super-parallel corpora of more than 1500+ unique languages for NLP research
☆32Updated last year
Related projects ⓘ
Alternatives and complementary repositories for 1000Langs
- NTREX -- News Test References for MT Evaluation☆75Updated 5 months ago
- These are lists for a variety of languages containing words that are distinctive to each language.☆34Updated 2 years ago
- A program to choose transfer languages for cross-lingual learning☆70Updated last year
- Statistics on multilingual datasets☆17Updated 2 years ago
- This repo contains a set of neural transducer, e.g. sequence-to-sequence model, focusing on character-level tasks.☆72Updated last year
- Python source code for EMNLP 2020 paper "Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT".☆35Updated 2 years ago
- Code for ACL 2022 paper "Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation"☆31Updated 2 years ago
- Code and data for the IWSLT 2022 shared task on Formality Control for SLT☆21Updated last year
- Tools for filtering and cleaning parallel and monolingual corpora for machine translation and other natural language processing tasks.☆41Updated 10 months ago
- ☆17Updated last year
- Multilingual Open Text☆25Updated last week
- A survey of corpora for Germanic low-resource languages and dialects☆24Updated 3 months ago
- ☆40Updated 2 years ago
- a tool for calcualting character n-gram F score☆66Updated last year
- ☆23Updated 4 years ago
- ParCourE - Parallel Corpus Explorer☆12Updated 2 years ago
- UDapter is a multilingual dependency parser that uses "contextual" adapters together with language-typology features for language-specifi…☆30Updated last year
- Code and data for: Low Resource Grammatical Error Correction Using Wikipedia Edits (WNUT 2018)☆12Updated 3 months ago
- Caucasus languages focused multilingual and monolingual corpuses for Natural Language Processing(NLP)☆31Updated last week
- OpusFilter - Parallel corpus processing toolkit☆102Updated 2 months ago
- ☆91Updated 8 months ago
- ☆50Updated 2 years ago
- Resource and Tool for Writing System Identification -- LREC 2024☆13Updated 5 months ago
- BERT models for many languages created from Wikipedia texts☆34Updated 4 years ago
- Library for experimenting with state-of-the-art evaluation metrics like UScore☆11Updated last year
- Zero-shot Transfer Learning from English to Arabic☆29Updated 2 years ago
- A library of translation-based text similarity measures☆25Updated 10 months ago
- This repositary hosts my experiments for the project, I did with OffNote Labs.☆11Updated 3 years ago
- Tool to fix bitexts and tag near-duplicates for removal☆29Updated 2 months ago
- A tiny BERT for low-resource monolingual models☆29Updated last month