bigscience-workshop / metadata
Experiments on including metadata such as URLs, timestamps, website descriptions and HTML tags during pretraining.
☆31Updated last year
Alternatives and similar repositories for metadata:
Users that are interested in metadata are comparing it to the libraries listed below
- ☆76Updated 3 years ago
- Apps built using Inspired Cognition's Critique.☆58Updated 2 years ago
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆93Updated 2 years ago
- ☆65Updated last year
- This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”☆85Updated 2 years ago
- ☆46Updated last year
- Pytorch Implementation of EncT5: Fine-tuning T5 Encoder for Non-autoregressive Tasks☆63Updated 3 years ago
- Official code and model checkpoints for our EMNLP 2022 paper "RankGen - Improving Text Generation with Large Ranking Models" (https://arx…☆136Updated last year
- ☆97Updated 2 years ago
- ☆58Updated 3 years ago
- ☆72Updated last year
- SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…☆48Updated last year
- Mr. TyDi is a multi-lingual benchmark dataset built on TyDi, covering eleven typologically diverse languages.☆75Updated 3 years ago
- Contrastive Fact Verification☆71Updated 2 years ago
- Research code for the paper "How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models"☆27Updated 3 years ago
- ☆51Updated last year
- ☆38Updated last year
- Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.☆80Updated 7 months ago
- Query-focused summarization data☆41Updated 2 years ago
- Shared code for training sentence embeddings with Flax / JAX☆27Updated 3 years ago
- Semantically Structured Sentence Embeddings☆66Updated 6 months ago
- Official repository for our EACL 2023 paper "LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization" (https…☆43Updated 8 months ago
- Code & data for EMNLP 2020 paper "MOCHA: A Dataset for Training and Evaluating Reading Comprehension Metrics".☆16Updated 3 years ago
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning☆30Updated 2 years ago
- MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization☆72Updated 3 years ago
- M2D2: A Massively Multi-domain Language Modeling Dataset (EMNLP 2022) by Machel Reid, Victor Zhong, Suchin Gururangan, Luke Zettlemoyer☆55Updated 2 years ago
- How Contextual are Contextualized Word Representations?☆41Updated 5 years ago
- Code and Data for Evaluation WG☆41Updated 3 years ago
- BLOOM+1: Adapting BLOOM model to support a new unseen language☆71Updated last year
- A Python Commonsense Knowledge Inference Toolkit☆64Updated last year