fdschmidt93 / trident-nllb-llm2vec
Repository for "Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages"
☆13Updated 4 months ago
Alternatives and similar repositories for trident-nllb-llm2vec:
Users that are interested in trident-nllb-llm2vec are comparing it to the libraries listed below
- Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…☆25Updated 10 months ago
- Prabhupadavani: A Code-mixed Speech Translation Data for 25 languages☆13Updated 2 years ago
- Repository containing the open source code of works published at the FBK MT unit.☆42Updated last month
- LTG-Bert☆29Updated last year
- A library for data streaming and augmentation☆20Updated 11 months ago
- ☆20Updated 2 years ago
- NTREX -- News Test References for MT Evaluation☆81Updated 8 months ago
- A library for minimum Bayes risk (MBR) decoding☆35Updated last week
- Multilingual Open Text☆25Updated 4 months ago
- Minimum Bayes Risk Decoding for Hugging Face Transformers☆55Updated 8 months ago
- This repository contains the implementation of the paper: "Span Classification with Structured Information for Disfluency Detection in Sp…☆12Updated last year
- Scaling Sparse Fine-Tuning to Large Language Models☆16Updated last year
- GlotCC Dataset and Pipline -- NeurIPS 2024☆17Updated 4 months ago
- phone inventory library☆16Updated last year
- Implementation of "SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages" paper, accepted to E…☆25Updated 2 years ago
- Final training script from HuggingFace Whisper Fine tuning event - to get best results on finetuned model.☆12Updated 2 years ago
- asr2k☆49Updated 8 months ago
- A tiny BERT for low-resource monolingual models☆31Updated 5 months ago
- ☆14Updated 4 months ago
- ☆16Updated this week
- Code for SaGe subword tokenizer (EACL 2023)☆24Updated 3 months ago
- ☆34Updated 3 years ago
- ☆51Updated last year
- Code for the paper-"Mirostat: A Perplexity-Controlled Neural Text Decoding Algorithm" (https://arxiv.org/abs/2007.14966).☆58Updated 3 years ago
- Source code for the GPT-2 story generation models in the EMNLP 2020 paper "STORIUM: A Dataset and Evaluation Platform for Human-in-the-Lo…☆39Updated last year
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning☆29Updated 2 years ago
- Can LLMs generate code-mixed sentences through zero-shot prompting?☆11Updated last year
- Code for the paper "Getting the most out of your tokenizer for pre-training and domain adaptation"☆16Updated last year
- Suite for phonetic word embeddings, especially their evaluation and baseline models.☆25Updated last week