fdschmidt93 / trident-nllb-llm2vecLinks
Repository for "Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages"
☆15Updated last year
Alternatives and similar repositories for trident-nllb-llm2vec
Users that are interested in trident-nllb-llm2vec are comparing it to the libraries listed below
Sorting:
- 🕸 GlotCC Dataset and Pipline -- NeurIPS 2024☆20Updated 10 months ago
- Official implementation of "GPT or BERT: why not both?"☆61Updated 6 months ago
- A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB te…☆295Updated last week
- BLOOM+1: Adapting BLOOM model to support a new unseen language☆74Updated last year
- NTREX -- News Test References for MT Evaluation☆88Updated last year
- OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.☆57Updated last week
- https://liuzeming01.github.io/XDailyDialog/☆13Updated 2 years ago
- Minimum Bayes Risk Decoding for Hugging Face Transformers☆60Updated last year
- Repository containing the open source code of works published at the FBK MT unit.☆59Updated 3 weeks ago
- A tiny BERT for low-resource monolingual models☆31Updated last month
- babyLM WhisBERT code☆19Updated last year
- Reduce the size of pretrained Hugging Face models via vocabulary trimming.☆48Updated 3 years ago
- Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback☆96Updated 2 years ago
- [TMLR'23] Contrastive Search Is What You Need For Neural Text Generation☆123Updated 2 years ago
- Facebook AI Research Sequence-to-Sequence Toolkit written in Python.☆25Updated 5 months ago
- Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.☆51Updated 9 months ago
- SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects☆23Updated last year
- Implementation of "SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages" paper, accepted to E…☆25Updated 3 years ago
- Repository for fine-tuning Transformers 🤗 based seq2seq speech models in JAX/Flax.☆38Updated 2 years ago
- Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023☆107Updated last year
- An official implementation of "BPE-Dropout: Simple and Effective Subword Regularization" algorithm.☆53Updated 4 years ago
- Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP☆58Updated 3 years ago
- A repository for experiments in quality-aware decoding☆18Updated 3 years ago
- SpanAlign: Sentence Alignment Method based on Cross-Language Span Prediction and ILP☆14Updated 4 years ago
- ☆21Updated 3 years ago
- Library for pruning experts per language pair in NLLB-200☆34Updated 2 years ago
- ☆34Updated 2 years ago
- ☆57Updated 3 years ago
- Code for Zero-Shot Tokenizer Transfer☆142Updated last year
- This tool helps automatic generation of grammatically valid synthetic Code-mixed data by utilizing linguistic theories such as Equivalenc…☆58Updated last year