Niccolo-Ajroldi / plainLM

Minimal pretraining script for language modeling in PyTorch. Supporting torch compilation and DDP. It includes a model implementation and a data preprocessing.
12Updated last week

Alternatives and similar repositories for plainLM:

Users that are interested in plainLM are comparing it to the libraries listed below