[EMNLP 2023] π¬ Language Identification with Support for More Than 2000 Labels
β229Nov 9, 2026Updated last week
Alternatives and similar repositories for GlotLID
Users that are interested in GlotLID are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- [LREC 2024] π Resource and Tool for Writing System Identificationβ25Nov 6, 2026Updated 2 weeks ago
- [WWW 2026] πΈ GlotWeb: Web Indexing for Minority Languagesβ17Apr 14, 2026Updated 7 months ago
- Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)β76Apr 1, 2025Updated last year
- [ACL 2023] Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagesβ109Apr 14, 2026Updated 7 months ago
- [NAACL 2024] A Framework aims to wisely initialize unseen subword embeddings in PLMs for efficient large-scale continued pretrainingβ18Nov 26, 2023Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways β’ AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- [ACL 2025] π Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignmentβ11Oct 31, 2026Updated 3 weeks ago
- A Multilingual Keyboard Layout-Based Typo Generatorβ19Nov 23, 2025Updated 11 months ago
- [EMNLP'23] Official Code for "FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models"β40Jun 7, 2025Updated last year
- SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectsβ28May 20, 2026Updated 6 months ago
- [NeurIPS 2024] πΈ GlotCC Dataset and Piplineβ21Apr 6, 2025Updated last year
- AfroLID, a powerful neural toolkit for African languages identification which covers 517 African languages.β40Feb 5, 2026Updated 9 months ago
- GlotEval: a unified evaluation toolkit designed to benchmark multilingual Large Language Models (LLMs) in a language-specific wayβ19Nov 4, 2025Updated last year
- β274Oct 27, 2025Updated last year
- mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Modelsβ11Jan 19, 2024Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform β’ AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- β154Jul 2, 2026Updated 4 months ago
- β13Aug 23, 2024Updated 2 years ago
- Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.β93Sep 12, 2024Updated 2 years ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.β3,429Updated this week
- The FLORES+ Machine Translation Benchmarkβ112Nov 12, 2024Updated 2 years ago
- Tool to fix bitexts and tag near-duplicates for removalβ36Sep 4, 2025Updated last year
- [ACL 2025] Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchuβ13Sep 27, 2025Updated last year
- Python source code for EMNLP 2021 Findings paper: "Subword Mapping and Anchoring Across Languages".β13Sep 17, 2021Updated 5 years ago
- Bicleaner fork that uses neural networksβ41Feb 23, 2026Updated 8 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI β’ AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB teβ¦β315Aug 16, 2026Updated 3 months ago
- Hengam: An Adversarially Trained Transformer for Persian Temporal Tagging (AACL'22)β11Aug 25, 2023Updated 3 years ago
- Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization [ACL 2026]β20Apr 18, 2026Updated 7 months ago
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learningβ31Jan 25, 2023Updated 3 years ago
- A different, but useful, textcat approach.β18Jul 15, 2024Updated 2 years ago
- Repository for the experiments in my paper accepted to the CLIN Journal: "Selecting Parallel In-domain Sentences for Neural Machine Transβ¦β11Nov 28, 2025Updated 11 months ago
- NTREX -- News Test References for MT Evaluationβ87Jun 5, 2024Updated 2 years ago
- Seed Machine Translation Dataβ34Nov 12, 2024Updated 2 years ago
- A collection of datasets for language model pretraining including scripts for downloading, preprocesssing, and sampling.β68Jul 29, 2024Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient β’ AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Goldfish: Monolingual language models for 350 languages.β29Mar 4, 2026Updated 8 months ago
- [NAACL 2025] Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languagesβ12May 31, 2024Updated 2 years ago
- FeedbackQA: Improving Question Answering Post-Deployment with Interactive Feedbackβ12Jul 13, 2022Updated 4 years ago
- Code for Zero-Shot Tokenizer Transferβ147Jan 14, 2025Updated last year
- βοΈ Sentence segmentation with wtpsplit's state-of-the-art Segment any Text (SaT) modelsβ44May 2, 2026Updated 6 months ago
- Just keeping an eye on the ecosystem.β24Updated this week
- [ACL 2024] π¬ MaskLID: Code-Switching Language Identification through Iterative Maskingβ13Nov 6, 2026Updated 2 weeks ago
Are these results useful?