EleutherAI / openwebtext2
☆89Updated 2 years ago
Alternatives and similar repositories for openwebtext2:
Users that are interested in openwebtext2 are comparing it to the libraries listed below
- Python tools for processing the stackexchange data dumps into a text dataset for Language Models☆81Updated last year
- ☆77Updated last year
- Open source library for few shot NLP☆77Updated last year
- ☆33Updated last year
- Pipeline for pulling and processing online language model pretraining data from the web☆175Updated last year
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆93Updated 2 years ago
- Tools for managing datasets for governance and training.☆82Updated 2 weeks ago
- ☆97Updated 2 years ago
- Techniques used to run BLOOM at inference in parallel☆37Updated 2 years ago
- [TMLR'23] Contrastive Search Is What You Need For Neural Text Generation☆119Updated last year
- ☆73Updated last year
- ☆96Updated 2 years ago
- Evaluation suite for large-scale language models.☆124Updated 3 years ago
- ☆110Updated 2 years ago
- A Benchmark Dataset for Understanding Disfluencies in Question Answering☆62Updated 3 years ago
- Experiments with generating opensource language model assistants☆97Updated last year
- XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale☆154Updated last year
- Script for downloading GitHub.☆90Updated 7 months ago
- This project studies the performance and robustness of language models and task-adaptation methods.☆144Updated 8 months ago
- An experimental implementation of the retrieval-enhanced language model☆74Updated 2 years ago
- Transformers at any scale☆41Updated last year
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆208Updated last year
- No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval☆28Updated 2 years ago
- ☆38Updated 10 months ago
- Implementation of Marge, Pre-training via Paraphrasing, in Pytorch☆75Updated 4 years ago
- The pipeline for the OSCAR corpus☆166Updated last year
- ☆67Updated 2 years ago
- Official repository with code and data accompanying the NAACL 2021 paper "Hurdles to Progress in Long-form Question Answering" (https://a…☆46Updated 2 years ago
- Code for the arXiv paper: "LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond"☆59Updated 3 weeks ago
- Code for the paper-"Mirostat: A Perplexity-Controlled Neural Text Decoding Algorithm" (https://arxiv.org/abs/2007.14966).☆58Updated 3 years ago