hpcaitech / ColossalAI-ExamplesLinks
Examples of training models with hybrid parallelism using ColossalAI
☆340Updated 2 years ago
Alternatives and similar repositories for ColossalAI-Examples
Users that are interested in ColossalAI-Examples are comparing it to the libraries listed below
Sorting:
- Scalable PaLM implementation of PyTorch☆189Updated 2 years ago
- Large-scale model inference.☆630Updated last year
- Performance benchmarking with ColossalAI☆39Updated 2 years ago
- LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training☆406Updated 2 weeks ago
- Fast Inference Solutions for BLOOM☆564Updated 7 months ago
- Efficient Inference for Big Models☆584Updated 2 years ago
- A unified tokenization tool for Images, Chinese and English.☆152Updated 2 years ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,391Updated last year
- Efficient Training (including pre-training and fine-tuning) for Big Models☆594Updated last week
- Models and examples built with OneFlow☆97Updated 7 months ago
- Sky Computing: Accelerating Geo-distributed Computing in Federated Learning☆91Updated 2 years ago
- ☆459Updated 11 months ago
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆220Updated last year
- Running BERT without Padding☆471Updated 3 years ago
- ParaGen is a PyTorch deep learning framework for parallel sequence generation.☆186Updated 2 years ago
- The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )☆220Updated 5 months ago
- Best practice for training LLaMA models in Megatron-LM☆656Updated last year
- A collection of models built with ColossalAI☆32Updated 2 years ago
- Collaborative Training of Large Language Models in an Efficient Way☆415Updated 9 months ago
- Ongoing research training transformer models at scale☆387Updated 9 months ago
- minichatgpt - To Train ChatGPT In 5 Minutes☆168Updated last year
- Official repository for LongChat and LongEval☆518Updated last year
- ☆543Updated 5 months ago
- Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.☆95Updated last year
- 📑 Dive into Big Model Training☆113Updated 2 years ago
- ☆411Updated last year
- GPTQ inference Triton kernel☆300Updated 2 years ago
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆313Updated 2 years ago
- Official implementation of TransNormerLLM: A Faster and Better LLM☆243Updated last year
- Efficient AI Inference & Serving☆469Updated last year