bigscience-workshop / data_toolingLinks
Tools for managing datasets for governance and training.
☆85Updated 2 months ago
Alternatives and similar repositories for data_tooling
Users that are interested in data_tooling are comparing it to the libraries listed below
Sorting:
- Pipeline for pulling and processing online language model pretraining data from the web☆177Updated 2 years ago
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆93Updated 2 years ago
- The pipeline for the OSCAR corpus☆171Updated last year
- Datasets collection and preprocessings framework for NLP extreme multitask learning☆185Updated 3 weeks ago
- ☆100Updated 2 years ago
- Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.☆82Updated 10 months ago
- ☆72Updated 2 years ago
- BLOOM+1: Adapting BLOOM model to support a new unseen language☆73Updated last year
- ☆66Updated last year
- ☆90Updated 3 years ago
- XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale☆155Updated last year
- ☆79Updated last year
- Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023☆103Updated last year
- Open source library for few shot NLP☆78Updated 2 years ago
- Our open source implementation of MiniLMv2 (https://aclanthology.org/2021.findings-acl.188)☆61Updated 2 years ago
- Experiments with generating opensource language model assistants☆97Updated 2 years ago
- ☆104Updated 7 months ago
- ☆182Updated 2 years ago
- This project studies the performance and robustness of language models and task-adaptation methods.☆150Updated last year
- A framework for few-shot evaluation of autoregressive language models.☆105Updated 2 years ago
- Load What You Need: Smaller Multilingual Transformers for Pytorch and TensorFlow 2.0.☆105Updated 3 years ago
- ☆86Updated 4 months ago
- Pretraining Efficiently on S2ORC!☆165Updated 9 months ago
- A Multilingual Replicable Instruction-Following Model☆94Updated 2 years ago
- Easy modernBERT fine-tuning and multi-task learning☆59Updated last month
- 💬 Language Identification with Support for More Than 2000 Labels -- EMNLP 2023☆147Updated 2 months ago
- Apps built using Inspired Cognition's Critique.☆58Updated 2 years ago
- Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP☆58Updated 3 years ago
- This repository contains the code for "Generating Datasets with Pretrained Language Models".☆188Updated 3 years ago
- [TMLR'23] Contrastive Search Is What You Need For Neural Text Generation☆119Updated 2 years ago