Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.
☆98Feb 9, 2023Updated 3 years ago
Alternatives and similar repositories for olm-training
Users that are interested in olm-training are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Pipeline for pulling and processing online language model pretraining data from the web☆180Jul 31, 2023Updated 3 years ago
- Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"☆59Jan 12, 2023Updated 3 years ago
- LTG-Bert☆35Jan 8, 2024Updated 2 years ago
- [ICML 2023] Exploring the Benefits of Training Expert Language Models over Instruction Tuning☆98Apr 26, 2023Updated 3 years ago
- ☆11Jan 20, 2025Updated last year
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Experiments for XLM-V Transformers Integeration☆13Feb 8, 2023Updated 3 years ago
- An open collection of implementation tips, tricks and resources for training large language models☆507Mar 8, 2023Updated 3 years ago
- A tiny BERT for low-resource monolingual models☆33Dec 24, 2025Updated 10 months ago
- ☆16Mar 3, 2024Updated 2 years ago
- Experiments for efforts to train a new and improved t5☆76Apr 15, 2024Updated 2 years ago
- A fast implementation of T5/UL2 in PyTorch using Flash Attention☆117Oct 30, 2025Updated last year
- ☆102Dec 17, 2022Updated 3 years ago
- Code repo for "Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers" (ACL 2023)☆22Nov 1, 2023Updated 3 years ago
- Plug-and-play Search Interfaces with Pyserini and Hugging Face☆30Aug 5, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Mar 12, 2024Updated 2 years ago
- ☆11Jun 2, 2022Updated 4 years ago
- 🚀🤗 A collection of templates for Hugging Face Spaces☆34Oct 9, 2023Updated 3 years ago
- This repository contains the code for paper Prompting ELECTRA Few-Shot Learning with Discriminative Pre-Trained Models.☆48Jun 7, 2022Updated 4 years ago
- Staged Training for Transformer Language Models☆33Mar 31, 2022Updated 4 years ago
- A Streamlit app to add structured tags to a dataset card☆23Jun 30, 2022Updated 4 years ago
- ☆30Sep 27, 2021Updated 5 years ago
- Code Roberta version of RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder☆10Mar 16, 2023Updated 3 years ago
- Scaling Data-Constrained Language Models☆352Jun 28, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆11Oct 15, 2024Updated 2 years ago
- Official code and model checkpoints for our EMNLP 2022 paper "RankGen - Improving Text Generation with Large Ranking Models" (https://arx…☆140Aug 2, 2023Updated 3 years ago
- **ARCHIVED** Filesystem interface to 🤗 Hub☆60Apr 6, 2023Updated 3 years ago
- ☆13Mar 27, 2020Updated 6 years ago
- A repository to get acquainted with basic training tasks in natural language processing and machine learning☆11Dec 27, 2023Updated 2 years ago
- Lite weight wrapper for the independent implementation of SPLADE++ models for search & retrieval pipelines. Models and Library created by…☆35Aug 24, 2024Updated 2 years ago
- Long-context pretrained encoder-decoder models☆97Oct 28, 2022Updated 4 years ago
- [ACL'24 Oral] Analysing The Impact of Sequence Composition on Language Model Pre-Training☆24Aug 18, 2024Updated 2 years ago
- ☆13May 8, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.☆93Sep 12, 2024Updated 2 years ago
- Cramming the training of a (BERT-type) language model into limited compute.☆1,370Jun 13, 2024Updated 2 years ago
- Code for training CamemBERTa, a French language model based on DeBERTa V3. Development is done here https://gitlab.inria.fr/almanach/Came…☆14Nov 18, 2024Updated 2 years ago
- A package for fine tuning of pretrained NLP transformers using Semi Supervised Learning☆14Oct 27, 2021Updated 5 years ago
- Adding new tasks to T0 without catastrophic forgetting☆33Oct 20, 2022Updated 4 years ago
- Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".☆225Dec 16, 2025Updated 11 months ago
- ☆53Jun 6, 2023Updated 3 years ago
Are these results useful?