sinaahmadi / PersoArabicLID
PALI: Language identification for Perso-Arabic Scripts
☆9Updated last year
Related projects ⓘ
Alternatives and complementary repositories for PersoArabicLID
- Repository for Findings of EMNLP 2020 "Context-aware Stand-alone Neural Spelling Correction"☆18Updated 3 years ago
- A tiny BERT for low-resource monolingual models☆29Updated last month
- Tool to fix bitexts and tag near-duplicates for removal☆29Updated 3 months ago
- Statistics on multilingual datasets☆17Updated 2 years ago
- Code and data for the IWSLT 2022 shared task on Formality Control for SLT☆21Updated last year
- Tools for managing datasets for governance and training.☆78Updated 3 weeks ago
- LTG-Bert☆29Updated 10 months ago
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆92Updated last year
- Load What You Need: Smaller Multilingual Transformers for Pytorch and TensorFlow 2.0.☆101Updated 2 years ago
- ☆73Updated 3 years ago
- ☆9Updated last year
- Neural network sequence labeling model☆11Updated 4 years ago
- ☆17Updated 2 years ago
- This repositary hosts my experiments for the project, I did with OffNote Labs.☆11Updated 3 years ago
- These are lists for a variety of languages containing words that are distinctive to each language.☆34Updated 2 years ago
- Code for the paper "Getting the most out of your tokenizer for pre-training and domain adaptation"☆13Updated 9 months ago
- ☆12Updated 3 years ago
- Library for fast text representation and classification.☆28Updated 10 months ago
- NTREX -- News Test References for MT Evaluation☆75Updated 5 months ago
- Multilingual Open Text☆25Updated 3 weeks ago
- Source code for the Apple reproduction☆31Updated 3 years ago
- GrammarTagger — A Neural Multilingual Grammar Profiler for Language Learning☆27Updated 3 years ago
- Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.☆75Updated 2 months ago
- BLOOM+1: Adapting BLOOM model to support a new unseen language☆70Updated 8 months ago
- A library for data streaming and augmentation☆20Updated 8 months ago
- ParaNames: A multilingual resource for parallel names☆30Updated 6 months ago
- Do Multilingual Language Models Think Better in English?☆41Updated last year
- Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP☆58Updated 2 years ago
- Using short models to classify long texts☆20Updated last year
- A library of translation-based text similarity measures☆25Updated 11 months ago