ymoslem / Adaptive-MT-LLM-Fine-tuning
Fine-tuning Open-Source LLMs for Adaptive Machine Translation
☆63Updated 4 months ago
Related projects ⓘ
Alternatives and complementary repositories for Adaptive-MT-LLM-Fine-tuning
- Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023☆96Updated 6 months ago
- A Multilingual Replicable Instruction-Following Model☆93Updated last year
- ☆219Updated 5 months ago
- GEMBA — GPT Estimation Metric Based Assessment☆100Updated 3 months ago
- Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback☆91Updated last year
- Multilingual Large Language Models Evaluation Benchmark☆105Updated 2 months ago
- BLOOM+1: Adapting BLOOM model to support a new unseen language☆70Updated 8 months ago
- NTREX -- News Test References for MT Evaluation☆75Updated 5 months ago
- Code for Multilingual Eval of Generative AI paper published at EMNLP 2023☆65Updated 8 months ago
- Language Identification with Support for More Than 2000 Labels -- EMNLP 2023☆89Updated last week
- A collection of preprocessed datasets and pretrained models for generating paraphrases.☆29Updated 3 years ago
- A collection of datasets for language model pretraining including scripts for downloading, preprocesssing, and sampling.☆52Updated 3 months ago
- The FLORES+ Machine Translation Benchmark☆99Updated this week
- A library of translation-based text similarity measures☆25Updated 11 months ago
- ☆62Updated 9 months ago
- Train Llama 2 & 3 on the SQuAD v2 task as an example of how to specialize a generalized (foundation) model.☆47Updated 5 months ago
- MAFAND-MT☆54Updated 4 months ago
- The implementation of "Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Deco…☆32Updated 9 months ago
- Tools for evaluating the performance of MT metrics on data from recent WMT metrics shared tasks.☆90Updated last month
- ☆147Updated 4 months ago
- ☆95Updated last year
- Domain-Specific Text Generation for Machine Translation (with LLMs) - scripts and config files for the paper☆15Updated last year
- Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.☆64Updated 3 weeks ago
- Official implementation of the paper "CoEdIT: Text Editing by Task-Specific Instruction Tuning" (EMNLP 2023)☆107Updated last month
- A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB te…☆250Updated last month
- Official implementations for (1) BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation and (2) Discourse Centric …☆71Updated last year
- MultilingualSIFT: Multilingual Supervised Instruction Fine-tuning☆86Updated last year
- Benchmarking Large Language Models☆80Updated last month
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆92Updated last year
- Tools for managing datasets for governance and training.☆77Updated 2 weeks ago