hpcaitech / ColossalAI-Examples
Examples of training models with hybrid parallelism using ColossalAI
☆337Updated last year
Alternatives and similar repositories for ColossalAI-Examples:
Users that are interested in ColossalAI-Examples are comparing it to the libraries listed below
- Scalable PaLM implementation of PyTorch☆192Updated 2 years ago
- Large-scale model inference.☆628Updated last year
- Performance benchmarking with ColossalAI☆39Updated 2 years ago
- LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training☆398Updated last month
- Efficient Inference for Big Models☆578Updated 2 years ago
- Fast Inference Solutions for BLOOM☆563Updated 4 months ago
- Collaborative Training of Large Language Models in an Efficient Way☆413Updated 6 months ago
- ☆456Updated 8 months ago
- Efficient Training (including pre-training and fine-tuning) for Big Models☆578Updated 7 months ago
- [NIPS2023] RRHF & Wombat☆804Updated last year
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆215Updated last year
- A unified tokenization tool for Images, Chinese and English.☆151Updated last year
- Best practice for training LLaMA models in Megatron-LM☆646Updated last year
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,368Updated 11 months ago
- Crosslingual Generalization through Multitask Finetuning☆528Updated 5 months ago
- ☆540Updated 2 months ago
- GPTQ inference Triton kernel☆297Updated last year
- Sky Computing: Accelerating Geo-distributed Computing in Federated Learning☆90Updated 2 years ago
- Open Academic Research on Improving LLaMA to SOTA LLM☆1,618Updated last year
- Running BERT without Padding☆470Updated 2 years ago
- Naive Bayes-based Context Extension☆320Updated 2 months ago
- EVA: Large-scale Pre-trained Chit-Chat Models☆307Updated last year
- SwissArmyTransformer is a flexible and powerful library to develop your own Transformer variants.☆1,058Updated 2 months ago
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆308Updated last year
- ☆306Updated last year
- Model Compression for Big Models☆157Updated last year
- ☆213Updated last year
- Introduction to CPM☆164Updated 3 years ago
- Microsoft Automatic Mixed Precision Library☆576Updated 5 months ago
- Implementation of Chinese ChatGPT☆287Updated last year