google-research-datasets / QAmeleon
QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning PaLM with only five examples per language. We use the synthetic data to finetune downstream QA models leading to improved accuracy in comparison to English-only and translation-based baselines.
☆34Updated last year
Alternatives and similar repositories for QAmeleon:
Users that are interested in QAmeleon are comparing it to the libraries listed below
- Embedding Recycling for Language models☆38Updated last year
- Codes and files for the paper Are Emergent Abilities in Large Language Models just In-Context Learning☆33Updated 3 months ago
- Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…☆26Updated 11 months ago
- QLoRA for Masked Language Modeling☆22Updated last year
- A library for squeakily cleaning and filtering language datasets.☆46Updated last year
- My explorations into editing the knowledge and memories of an attention network☆34Updated 2 years ago
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Updated last year
- ☆44Updated 4 months ago
- ☆38Updated 11 months ago
- This repository contains code for cleaning your training data of benchmark data to help combat data snooping.☆25Updated last year
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆93Updated 2 years ago
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…☆14Updated last year
- ☆14Updated 6 months ago
- ☆11Updated 4 months ago
- Minimum Bayes Risk Decoding for Hugging Face Transformers☆57Updated 10 months ago
- Experiments with generating opensource language model assistants☆97Updated last year
- See https://github.com/cuda-mode/triton-index/ instead!☆11Updated 11 months ago
- Plug-and-play Search Interfaces with Pyserini and Hugging Face☆31Updated last year
- SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…☆48Updated last year
- Index of URLs to pdf files all over the internet and scripts☆23Updated last year
- Ranking of fine-tuned HF models as base models.☆35Updated last year
- Using short models to classify long texts☆21Updated 2 years ago
- Code for paper "Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs"☆28Updated 2 years ago
- Aioli: A unified optimization framework for language model data mixing☆23Updated 2 months ago
- ☆67Updated 7 months ago
- ☆24Updated last year
- Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model☆42Updated last year
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated last year
- Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP☆58Updated 2 years ago
- ☆19Updated last year