naver / nllb-pruning
Library for pruning experts per language pair in NLLB-200
☆27Updated last year
Related projects ⓘ
Alternatives and complementary repositories for nllb-pruning
- Repository containing the open source code of works published at the FBK MT unit.☆42Updated 4 months ago
- Bicleaner fork that uses neural networks☆38Updated 3 months ago
- Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.☆75Updated 2 months ago
- Reduce the size of pretrained Hugging Face models via vocabulary trimming.☆43Updated last year
- A toolkit for Spoken Language Understanding Evaluation (SLUE) benchmark. Refer paper https://arxiv.org/abs/2111.10367 for more details. O…☆63Updated 8 months ago
- Curriculum training☆16Updated 2 months ago
- Complimentary code for our paper Automatic punctuation restoration with BERT models☆48Updated last year
- SiLLM is a Simultaneous Machine Translation (SiMT) Framework. It utilizes a Large Language model as the translation model and employs a t…☆15Updated 9 months ago
- Implementation of "SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages" paper, accepted to E…☆19Updated 2 years ago
- SHAS: Approaching optimal Segmentation for End-to-End Speech Translation☆37Updated last year
- NTREX -- News Test References for MT Evaluation☆75Updated 5 months ago
- Tool to fix bitexts and tag near-duplicates for removal☆29Updated 3 months ago
- BLOOM+1: Adapting BLOOM model to support a new unseen language☆70Updated 8 months ago
- Official code for Wav2Seq☆95Updated 2 years ago
- ☆28Updated 2 years ago
- A library for minimum Bayes risk (MBR) decoding☆31Updated this week
- Various speech datasets made available to the public☆99Updated last month
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning☆29Updated last year
- Explore different way to mix speech model(wav2vec2, hubert) and nlp model(BART,T5,GPT) together☆42Updated last year
- Incorporating KenLM language model with HuggingFace implementation of Wav2Vec2CTC Model using beam search decoding☆71Updated 3 years ago
- Unsupervised spoken sentence embeddings☆14Updated last year
- Language Identification with Support for More Than 2000 Labels -- EMNLP 2023☆90Updated 3 weeks ago
- This repository contains a demonstrative implementation for pooling-based models, e.g., DeepPyramidion complementing our paper "Sparsifyi…☆14Updated 2 years ago
- ☆9Updated last year
- ☆33Updated 3 years ago
- asr2k☆48Updated 5 months ago
- Code and data for the IWSLT 2022 shared task on Formality Control for SLT☆21Updated last year
- Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023☆96Updated 7 months ago
- Facebook AI Research Sequence-to-Sequence Toolkit written in Python.☆22Updated 3 weeks ago