facebookresearch / belebele
Repo for the Belebele dataset, a massively multilingual reading comprehension dataset.
☆321Updated 3 months ago
Alternatives and similar repositories for belebele:
Users that are interested in belebele are comparing it to the libraries listed below
- Manage scalable open LLM inference endpoints in Slurm clusters☆253Updated 8 months ago
- ☆504Updated 4 months ago
- ☆359Updated last year
- Fast & more realistic evaluation of chat language models. Includes leaderboard.☆185Updated last year
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆300Updated last year
- Pipeline for pulling and processing online language model pretraining data from the web☆177Updated last year
- Automatically evaluate your LLMs in Google Colab☆613Updated 10 months ago
- A bagel, with everything.☆317Updated 11 months ago
- FastFit ⚡ When LLMs are Unfit Use FastFit ⚡ Fast and Effective Text Classification with Many Classes☆191Updated 5 months ago
- A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB te…☆271Updated 2 months ago
- Code for fine-tuning Platypus fam LLMs using LoRA☆628Updated last year
- Build, evaluate, understand, and fix LLM-based apps☆487Updated last year
- This is the repo for the paper Shepherd -- A Critic for Language Model Generation☆218Updated last year
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆208Updated last year
- Generate textbook-quality synthetic LLM pretraining data☆498Updated last year
- A large-scale multilingual dataset for Information Retrieval. Thorough human-annotations across 18 diverse languages.☆180Updated 8 months ago
- Generative Representational Instruction Tuning☆613Updated 2 weeks ago
- experiments with inference on llama☆104Updated 9 months ago
- Datasets collection and preprocessings framework for NLP extreme multitask learning☆176Updated 2 months ago
- Website for hosting the Open Foundation Models Cheat Sheet.☆265Updated 3 weeks ago
- The pipeline for the OSCAR corpus☆167Updated last year
- Fully fine-tune large models like Mistral, Llama-2-13B, or Qwen-14B completely for free☆230Updated 5 months ago
- Notus is a collection of fine-tuned LLMs using SFT, DPO, SFT+DPO, and/or any other RLHF techniques, while always keeping a data-first app…☆166Updated last year
- batched loras☆340Updated last year
- ☆449Updated last year
- Let's build better datasets, together!☆257Updated 3 months ago
- Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.☆73Updated 5 months ago
- Domain Adapted Language Modeling Toolkit - E2E RAG☆318Updated 4 months ago
- 💬 Language Identification with Support for More Than 2000 Labels -- EMNLP 2023☆124Updated 4 months ago
- Extend existing LLMs way beyond the original training length with constant memory usage, without retraining☆692Updated 11 months ago