Code and data for the paper "Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed?"
☆26Jun 3, 2025Updated 9 months ago
Alternatives and similar repositories for multilingual-instruction-tuning
Users that are interested in multilingual-instruction-tuning are comparing it to the libraries listed below
Sorting:
- ☆13Aug 23, 2024Updated last year
- Curriculum training☆22Jun 25, 2025Updated 8 months ago
- SiLLM is a Simultaneous Machine Translation (SiMT) Framework. It utilizes a Large Language model as the translation model and employs a t…☆18Feb 22, 2024Updated 2 years ago
- Library for experimenting with state-of-the-art evaluation metrics like UScore☆12May 27, 2023Updated 2 years ago
- mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models☆11Jan 19, 2024Updated 2 years ago
- 🔍 Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment☆11Apr 6, 2025Updated 11 months ago
- ☆21Dec 5, 2022Updated 3 years ago
- Evaluation results for Machine Translation within the BigScience project☆11May 15, 2023Updated 2 years ago
- Instruction Following Eval☆16Jan 16, 2025Updated last year
- Code for NeurIPS 2023 paper "Non-autoregressive Machine Translation with Probabilistic Context-free Grammar".☆12Jan 4, 2024Updated 2 years ago
- Official code for the NeurIPS25 paper "RAT: Bridging RNN Efficiencyand Attention Accuracy in Language Modeling" (https://arxiv.org/abs/25…☆23Dec 10, 2025Updated 2 months ago
- [ACL 2024] An easily extensible framework for simultaneous, text-to-text neural machine translation (SimulMT) for LLMs.☆18Apr 21, 2025Updated 10 months ago
- A library for minimum Bayes risk (MBR) decoding☆52Nov 2, 2025Updated 4 months ago
- A lightweight, user-friendly data-plane for LLM training.☆38Sep 10, 2025Updated 5 months ago
- Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023☆106Apr 20, 2024Updated last year
- Easy-to-use framework for evaluating cross-lingual consistency of factual knowledge (Supported LLaMA, BLOOM, mT5, RoBERTa, etc.) Paper he…☆27Aug 8, 2025Updated 7 months ago
- ☆21Feb 13, 2023Updated 3 years ago
- Official code release for "SuperBPE: Space Travel for Language Models"☆89Jan 9, 2026Updated 2 months ago
- KoCommonGEN v2: A Benchmark for Navigating Korean Commonsense Reasoning Challenges in Large Language Models☆25Aug 24, 2024Updated last year
- An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset☆28Jan 19, 2025Updated last year
- Official Code for M-RᴇᴡᴀʀᴅBᴇɴᴄʜ: Evaluating Reward Models in Multilingual Settings (ACL 2025 Main)☆41May 16, 2025Updated 9 months ago
- NAACL 2024: SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning☆26Mar 3, 2025Updated last year
- Tools for formatting WMT hypothesis and test sets in XML☆27Apr 18, 2025Updated 10 months ago
- ☆36Nov 15, 2023Updated 2 years ago
- The implementation of "Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Deco…☆36Aug 29, 2025Updated 6 months ago
- UDapter is a multilingual dependency parser that uses "contextual" adapters together with language-typology features for language-specifi…☆31Dec 5, 2022Updated 3 years ago
- Code for Multilingual Eval of Generative AI paper published at EMNLP 2023☆72Mar 6, 2024Updated 2 years ago
- ☆35Jun 15, 2023Updated 2 years ago
- 日本語マルチタスク言語理解ベンチマーク Japanese Massive Multitask Language Understanding Benchmark☆38Oct 7, 2025Updated 5 months ago
- Do Multilingual Language Models Think Better in English?☆42Aug 3, 2023Updated 2 years ago
- TOD-Flow: Modeling the Structure of Task-Oriented Dialogues☆13Feb 7, 2024Updated 2 years ago
- ☆39Jan 23, 2024Updated 2 years ago
- ☆10Oct 2, 2024Updated last year
- ☆12Jan 15, 2015Updated 11 years ago
- A library for evaluation of Grammatical Error Correction (GEC). Accepted to ACL'25 Demo: "gec-metrics: A Unified Library for Grammatical …☆14Jan 25, 2026Updated last month
- Modified version of fairseq, including new implementations for criterions using reinforcement learning methods.☆11Aug 14, 2019Updated 6 years ago
- Linear Attention for Efficient Bidirectional Sequence Modeling☆15May 13, 2025Updated 9 months ago
- ☆11May 18, 2022Updated 3 years ago
- Repository containing the open source code of works published at the FBK MT unit.☆60Jan 16, 2026Updated last month