cisnlp / GlotCCLinks
🕸 GlotCC Dataset and Pipline -- NeurIPS 2024
☆19Updated 2 months ago
Alternatives and similar repositories for GlotCC
Users that are interested in GlotCC are comparing it to the libraries listed below
Sorting:
- BPE modification that implements removing of the intermediate tokens during tokenizer training.☆25Updated 7 months ago
- 🔍 Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment☆11Updated 2 months ago
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated last year
- Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…☆27Updated last year
- 🚀🤗 A collection of templates for Hugging Face Spaces☆35Updated last year
- ☆12Updated 6 months ago
- QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…☆34Updated last year
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Updated last year
- Using short models to classify long texts☆21Updated 2 years ago
- ☆13Updated last month
- Official repository for "BLEUBERI: BLEU is a surprisingly effective reward for instruction following"☆23Updated 3 weeks ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆57Updated 9 months ago
- Trully flash implementation of DeBERTa disentangled attention mechanism.☆59Updated last month
- A toolkit implementing advanced methods to transfer models and model knowledge across tokenizers.☆27Updated last month
- Code for the paper "Getting the most out of your tokenizer for pre-training and domain adaptation"☆19Updated last year
- Code, results and other artifacts from the paper introducing the WildChat-50m dataset and the Re-Wild model family.☆29Updated 2 months ago
- ☆47Updated 4 months ago
- Code for SaGe subword tokenizer (EACL 2023)☆25Updated 6 months ago
- Plug-and-play Search Interfaces with Pyserini and Hugging Face☆32Updated last year
- ☆38Updated last year
- ☆25Updated last year
- MergeBench: A Benchmark for Merging Domain-Specialized LLMs☆14Updated last month
- Small python package to measure OCR quality and other related metrics.☆23Updated last year
- MEXMA: Token-level objectives improve sentence representations☆41Updated 5 months ago
- SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…☆48Updated last year
- Official Repository for "Hypencoder: Hypernetworks for Information Retrieval"☆25Updated 3 months ago
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning☆30Updated 2 years ago
- Starbucks: Improved Training for 2D Matryoshka Embeddings☆21Updated 4 months ago
- A package dedicated for running benchmark agreement testing☆16Updated last month
- Using open source LLMs to build synthetic datasets for direct preference optimization☆64Updated last year