bigscience-workshop / bigscienceLinks
Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
☆1,000Updated 10 months ago
Alternatives and similar repositories for bigscience
Users that are interested in bigscience are comparing it to the libraries listed below
Sorting:
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,397Updated last year
- Fast Inference Solutions for BLOOM☆564Updated 8 months ago
- Expanding natural instructions☆1,006Updated last year
- ☆1,527Updated last week
- distributed trainer for LLMs☆577Updated last year
- Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)☆464Updated 2 years ago
- Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀☆1,688Updated 8 months ago
- Crosslingual Generalization through Multitask Finetuning☆536Updated 9 months ago
- Parallelformers: An Efficient Model Parallelization Toolkit for Deployment☆790Updated 2 years ago
- Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch☆866Updated last year
- ☆1,224Updated 10 months ago
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆313Updated 2 years ago
- PyTorch extensions for high performance and large scale training.☆3,335Updated 2 months ago
- Tools to download and cleanup Common Crawl data☆1,016Updated 2 years ago
- ☆1,575Updated 2 years ago
- An open collection of implementation tips, tricks and resources for training large language models☆475Updated 2 years ago
- Mistral: A strong, northwesterly wind: Framework for transparent and accessible large-scale language model training, built with Hugging F…☆573Updated last year
- Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackab…☆1,572Updated last year
- Cramming the training of a (BERT-type) language model into limited compute.☆1,336Updated last year
- A modular RL library to fine-tune language models to human preferences☆2,317Updated last year
- Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".☆2,131Updated last year
- Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways☆823Updated 2 years ago
- Task-based datasets, preprocessing, and evaluation for sequence models.☆582Updated last month
- The hub for EleutherAI's work on interpretability and learning dynamics☆2,547Updated 2 weeks ago
- Library for 8-bit optimizers and quantization routines.☆715Updated 2 years ago
- Code repository for supporting the paper "Atlas Few-shot Learning with Retrieval Augmented Language Models",(https//arxiv.org/abs/2208.03…☆539Updated last year
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆2,091Updated 3 months ago
- All-in-one text de-duplication☆690Updated last month
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆2,020Updated 3 months ago
- FastFormers - highly efficient transformer models for NLU☆705Updated 3 months ago