Code and data for the paper "Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed?"
☆26Jun 3, 2025Updated last year
Alternatives and similar repositories for multilingual-instruction-tuning
Users that are interested in multilingual-instruction-tuning are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Curriculum training☆24Jun 25, 2025Updated last year
- SiLLM is a Simultaneous Machine Translation (SiMT) Framework. It utilizes a Large Language model as the translation model and employs a t…☆18Feb 22, 2024Updated 2 years ago
- [ACL 2025] 🔍 Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment☆11Apr 6, 2025Updated last year
- Evaluation results for Machine Translation within the BigScience project☆11May 15, 2023Updated 3 years ago
- Minimum Bayes Risk Decoding for Hugging Face Transformers☆61Jun 3, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization [ACL 2026]☆20Apr 18, 2026Updated 3 months ago
- Instruction Following Eval☆18Jan 16, 2025Updated last year
- NAACL 2024: SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning☆26Mar 3, 2025Updated last year
- Tutorials on how to Query Data☆13Jan 7, 2023Updated 3 years ago
- [ICML'25] "Rethinking Addressing in Language Models via Contextualized Equivariant Positional Encoding" by Jiajun Zhu, Peihao Wang, Ruisi…☆15Jun 6, 2025Updated last year
- A package containing utils for the PyTorch version of the Tapas algorithm.☆11Apr 29, 2021Updated 5 years ago
- Code for EMNLP 2022 main conference paper "Information-Transport-based Policy for Simultaneous Translation"☆13Nov 3, 2022Updated 3 years ago
- A library for minimum Bayes risk (MBR) decoding☆53Nov 2, 2025Updated 9 months ago
- Find informative examples to efficiently (human)-evaluate NLG models.☆17Apr 22, 2026Updated 3 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆21Dec 5, 2022Updated 3 years ago
- ☆17Updated this week
- [ACL 2024] An easily extensible framework for simultaneous, text-to-text neural machine translation (SimulMT) for LLMs.☆18Apr 21, 2025Updated last year
- Python Web Scraper☆14Mar 7, 2018Updated 8 years ago
- [ACL 2023] Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages☆107Apr 14, 2026Updated 4 months ago
- PyTorch code for DeepTime: Deep Time-Index Meta-Learning for Non-Stationary Time-Series Forecasting☆11Jan 9, 2023Updated 3 years ago
- An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset☆27Jan 19, 2025Updated last year
- 日本語マルチタスク言語理解ベンチマーク Japanese Massive Multitask Language Understanding Benchmark☆41Oct 7, 2025Updated 10 months ago
- Adaptation datasets and scripts for the paper "Reducing gender bias in Neural Machine Translation as a domain adaptation problem" (ACL 20…☆13Mar 18, 2021Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆26Feb 27, 2022Updated 4 years ago
- ☆21Feb 13, 2023Updated 3 years ago
- A LangGraph-powered agent that finds and analyzes similar companies, leveraging Qdrant for data storage, Exa for research, and Gmail for …☆20Feb 26, 2026Updated 5 months ago
- ☆39Jan 23, 2024Updated 2 years ago
- [WMT 2022] Implementation of TAL-SJTU's system for WMT22 English-Livonian☆23May 4, 2023Updated 3 years ago
- Hands-on with popular deep learning datasets and tasks☆13Apr 4, 2023Updated 3 years ago
- Steering Vector Repo from "Extracting Latent Steering Vectors from Pretrained Language Models" - ACL2022 Findings☆11Mar 14, 2022Updated 4 years ago
- Hackbright Capstone Project☆11Apr 14, 2016Updated 10 years ago
- The implementation of "Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Deco…☆38Aug 29, 2025Updated 11 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Official code for the NeurIPS25 paper "RAT: Bridging RNN Efficiencyand Attention Accuracy in Language Modeling" (https://arxiv.org/abs/25…☆26Dec 10, 2025Updated 8 months ago
- Tools for formatting WMT hypothesis and test sets in XML☆27Apr 18, 2025Updated last year
- Easy-to-use framework for evaluating cross-lingual consistency of factual knowledge (Supported LLaMA, BLOOM, mT5, RoBERTa, etc.) Paper he…☆28Aug 8, 2025Updated last year
- EMNLP 2020: Filtering before Iteratively Referring for Knowledge-Grounded Response Selection in Retrieval-Based Chatbots☆12Dec 15, 2020Updated 5 years ago
- Code and pretrained models for "DUB: Discrete Unit Back-translation for Speech Translation" (ACL 2023 Findings)☆29Jun 28, 2023Updated 3 years ago
- ☆10Oct 17, 2021Updated 4 years ago
- xLM is a modular, research-friendly framework for developing and comparing non-autoregressive language models. Built on PyTorch and PyTor…☆28Jul 30, 2026Updated 2 weeks ago