cisnlp / GlotCCLinks
πΈ GlotCC Dataset and Pipline -- NeurIPS 2024
β19Updated last month
Alternatives and similar repositories for GlotCC
Users that are interested in GlotCC are comparing it to the libraries listed below
Sorting:
- Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/β¦β26Updated last year
- MEXMA: Token-level objectives improve sentence representationsβ41Updated 4 months ago
- π Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignmentβ11Updated last month
- Repository containing the SPIN experiments on the DIBT 10k ranked promptsβ24Updated last year
- Using short models to classify long textsβ21Updated 2 years ago
- Trully flash implementation of DeBERTa disentangled attention mechanism.β55Updated 2 weeks ago
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found heβ¦β31Updated last year
- β13Updated 2 weeks ago
- QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning Pβ¦β34Updated last year
- Code, results and other artifacts from the paper introducing the WildChat-50m dataset and the Re-Wild model family.β29Updated 2 months ago
- ππ€ A collection of templates for Hugging Face Spacesβ35Updated last year
- Official Repository for "Hypencoder: Hypernetworks for Information Retrieval"β25Updated 2 months ago
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learningβ30Updated 2 years ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignmentβ57Updated 9 months ago
- Code for SaGe subword tokenizer (EACL 2023)β25Updated 6 months ago
- β45Updated 4 months ago
- β12Updated 5 months ago
- Plug-and-play Search Interfaces with Pyserini and Hugging Faceβ32Updated last year
- BPE modification that implements removing of the intermediate tokens during tokenizer training.β25Updated 6 months ago
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" givenβ¦β14Updated last year
- LTG-Bertβ32Updated last year
- Repository for "Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages"β14Updated 8 months ago
- A toolkit implementing advanced methods to transfer models and model knowledge across tokenizers.β24Updated 3 weeks ago
- β22Updated 4 months ago
- Aioli: A unified optimization framework for language model data mixingβ25Updated 4 months ago
- SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 laβ¦β48Updated last year
- Embedding Recycling for Language modelsβ38Updated last year
- Code for Zero-Shot Tokenizer Transferβ128Updated 4 months ago
- Code repo for "Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers" (ACL 2023)β22Updated last year
- Code for the paper "Getting the most out of your tokenizer for pre-training and domain adaptation"β18Updated last year