Repo for the Belebele dataset, a massively multilingual reading comprehension dataset.
☆340Dec 18, 2024Updated last year
Alternatives and similar repositories for belebele
Users that are interested in belebele are comparing it to the libraries listed below
Sorting:
- ☆19Sep 16, 2025Updated 5 months ago
- 🕸 GlotCC Dataset and Pipline -- NeurIPS 2024☆20Apr 6, 2025Updated 10 months ago
- Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback☆96Aug 18, 2023Updated 2 years ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,915Updated this week
- ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward exp…☆226Sep 18, 2025Updated 5 months ago
- Flacuna was developed by fine-tuning Vicuna on Flan-mini, a comprehensive instruction collection encompassing various tasks. Vicuna is al…☆111Sep 10, 2023Updated 2 years ago
- Multilingual Large Language Models Evaluation Benchmark☆132Aug 21, 2024Updated last year
- COMET for African languages☆10Jan 24, 2025Updated last year
- ☆10Oct 2, 2024Updated last year
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Aug 25, 2023Updated 2 years ago
- ☆266Aug 1, 2025Updated 7 months ago
- Python intefrace for evaluation on chatgpt models☆19Feb 13, 2024Updated 2 years ago
- Code for fine-tuning Platypus fam LLMs using LoRA☆629Feb 4, 2024Updated 2 years ago
- Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023☆106Apr 20, 2024Updated last year
- [ICLR 2024] Efficient Streaming Language Models with Attention Sinks☆7,187Jul 11, 2024Updated last year
- Data and tools for generating and inspecting OLMo pre-training data.☆1,416Nov 5, 2025Updated 3 months ago
- Robust recipes to align language models with human and AI preferences☆5,510Sep 8, 2025Updated 5 months ago
- NTREX -- News Test References for MT Evaluation☆88Jun 5, 2024Updated last year
- Seamless Voice Interactions with LLMs☆12Oct 28, 2023Updated 2 years ago
- This repository contains code and tooling for the Abacus.AI LLM Context Expansion project. Also included are evaluation scripts and bench…☆600Nov 17, 2023Updated 2 years ago
- ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…☆2,175Oct 8, 2024Updated last year
- Code for ACL 2022 paper "Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation"☆30Apr 2, 2022Updated 3 years ago
- A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.☆32Sep 19, 2025Updated 5 months ago
- [ICLR 2024] Lemur: Open Foundation Models for Language Agents☆555Oct 28, 2023Updated 2 years ago
- 🔮 LLM GPU Calculator☆21Aug 19, 2023Updated 2 years ago
- Microsoft Automatic Mixed Precision Library☆636Dec 1, 2025Updated 3 months ago
- SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…☆49Nov 13, 2023Updated 2 years ago
- Facebook Low Resource (FLoRes) MT Benchmark☆766Nov 20, 2023Updated 2 years ago
- QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…☆35Aug 15, 2023Updated 2 years ago
- Stanford NLP Python library for Representation Finetuning (ReFT)☆1,560Jan 14, 2026Updated last month
- Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…☆28Apr 17, 2024Updated last year
- 💬 Language Identification with Support for More Than 2000 Labels -- EMNLP 2023☆188Nov 19, 2025Updated 3 months ago
- In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning☆35Aug 9, 2023Updated 2 years ago
- Code for Zero-Shot Tokenizer Transfer☆143Jan 14, 2025Updated last year
- LLM as a Chatbot Service☆3,332Nov 20, 2023Updated 2 years ago
- Salesforce open-source LLMs with 8k sequence length.☆725Jan 31, 2025Updated last year
- MPI Code Generation through Domain-Specific Language Models☆14Nov 19, 2024Updated last year
- CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation☆14Aug 19, 2025Updated 6 months ago
- ParaNames: A multilingual resource for parallel names☆39May 20, 2024Updated last year