allenai / smashed

SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.
33Updated 10 months ago

Alternatives and similar repositories for smashed:

Users that are interested in smashed are comparing it to the libraries listed below