hpcaitech / ColossalAI-Examples
Examples of training models with hybrid parallelism using ColossalAI
☆336Updated last year
Related projects ⓘ
Alternatives and complementary repositories for ColossalAI-Examples
- Scalable PaLM implementation of PyTorch☆192Updated last year
- Large-scale model inference.☆630Updated last year
- LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training☆390Updated this week
- Performance benchmarking with ColossalAI☆39Updated 2 years ago
- A unified tokenization tool for Images, Chinese and English.☆150Updated last year
- Efficient Training (including pre-training and fine-tuning) for Big Models☆564Updated 3 months ago
- Fast Inference Solutions for BLOOM☆560Updated last month
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,338Updated 8 months ago
- The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )☆212Updated 6 months ago
- Official implementation of TransNormerLLM: A Faster and Better LLM☆229Updated 9 months ago
- Efficient Inference for Big Models☆571Updated last year
- Code for the ALiBi method for transformer language models (ICLR 2022)☆507Updated last year
- Collaborative Training of Large Language Models in an Efficient Way☆411Updated 2 months ago
- Tutel MoE: An Optimized Mixture-of-Experts Implementation☆735Updated this week
- GPTQ inference Triton kernel☆284Updated last year
- Best practice for training LLaMA models in Megatron-LM☆628Updated 10 months ago
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆207Updated last year
- ☆209Updated last year
- ☆453Updated 5 months ago
- [NIPS2023] RRHF & Wombat☆798Updated last year
- Automatically split your PyTorch models on multiple GPUs for training & inference☆626Updated 10 months ago
- Official repository for LongChat and LongEval☆512Updated 5 months ago
- Microsoft Automatic Mixed Precision Library☆525Updated last month
- Running BERT without Padding☆460Updated 2 years ago
- This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/…☆96Updated 8 months ago
- ☆82Updated last year
- Sky Computing: Accelerating Geo-distributed Computing in Federated Learning☆90Updated last year
- Crosslingual Generalization through Multitask Finetuning☆516Updated last month
- minichatgpt - To Train ChatGPT In 5 Minutes☆167Updated last year
- Introduction to CPM☆163Updated 3 years ago