cisnlp / GlotCCLinks
🕸 GlotCC Dataset and Pipline -- NeurIPS 2024
☆20Updated 6 months ago
Alternatives and similar repositories for GlotCC
Users that are interested in GlotCC are comparing it to the libraries listed below
Sorting:
- A toolkit implementing advanced methods to transfer models and model knowledge across tokenizers.☆46Updated 3 months ago
- 🔍 Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment☆11Updated 6 months ago
- ☆22Updated 8 months ago
- ☆14Updated last month
- Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…☆27Updated last year
- BLOOM+1: Adapting BLOOM model to support a new unseen language☆74Updated last year
- Minimum Bayes Risk Decoding for Hugging Face Transformers☆60Updated last year
- Code for Zero-Shot Tokenizer Transfer☆138Updated 9 months ago
- MEXMA: Token-level objectives improve sentence representations☆41Updated 9 months ago
- ☆44Updated 11 months ago
- Using open source LLMs to build synthetic datasets for direct preference optimization☆66Updated last year
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆60Updated last year
- BPE modification that implements removing of the intermediate tokens during tokenizer training.☆25Updated 10 months ago
- Using short models to classify long texts☆21Updated 2 years ago
- Synthetic Data Generation for Evaluation☆13Updated 7 months ago
- ☆52Updated 8 months ago
- Trully flash implementation of DeBERTa disentangled attention mechanism.☆66Updated 3 weeks ago
- ☆13Updated 10 months ago
- QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…☆34Updated 2 years ago
- SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects☆22Updated 8 months ago
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated last year
- Repository for "I am a Strange Dataset: Metalinguistic Tests for Language Models"☆44Updated last year
- A collection of datasets for language model pretraining including scripts for downloading, preprocesssing, and sampling.☆61Updated last year
- ☆39Updated last year
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning☆30Updated 2 years ago
- Codes and files for the paper Are Emergent Abilities in Large Language Models just In-Context Learning☆33Updated 9 months ago
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆96Updated 2 years ago
- ☆26Updated 2 years ago
- ☆21Updated 2 years ago
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Updated 2 years ago