microsoft / deepspeed-gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
☆19Updated last year
Related projects ⓘ
Alternatives and complementary repositories for deepspeed-gpt-neox
- Megatron LM 11B on Huggingface Transformers☆27Updated 3 years ago
- Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI☆58Updated last year
- Implementation of autoregressive language model using improved Transformer and DeepSpeed pipeline parallelism.☆32Updated 2 years ago
- ☆23Updated 2 years ago
- Implementation of stop sequencer for Huggingface Transformers☆15Updated last year
- Train 🤗transformers with DeepSpeed: ZeRO-2, ZeRO-3☆21Updated 3 years ago
- MeCab model trained with OpenKorPos.☆22Updated 2 years ago
- ↔️ T5 Machine Translation from English to Korean☆17Updated 2 years ago
- ☆31Updated 2 years ago
- URL downloader supporting checkpointing and continuous checksumming.☆19Updated 11 months ago
- 🤗 Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch.☆13Updated last week
- Repo for "Smart Word Suggestions" (SWS) task and benchmark☆19Updated 11 months ago
- Deploy KoGPT with Triton Inference Server☆14Updated 2 years ago
- A PyTorch Implementation of the Luna: Linear Unified Nested Attention☆41Updated 3 years ago
- data related codebase for polyglot project☆19Updated last year
- ☆86Updated 2 years ago
- ☆9Updated last week
- Convenient Text-to-Text Training for Transformers☆19Updated 2 years ago
- Implementation of N-Grammer, augmenting Transformers with latent n-grams, in Pytorch☆72Updated last year
- ☆19Updated 3 years ago
- ☆30Updated 2 years ago
- Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP☆58Updated 2 years ago
- ☆19Updated 2 years ago
- The official repo of our research work "Interactive Editing for Text Summarization".☆22Updated last year
- Official code for "Distributed Deep Learning in Open Collaborations" (NeurIPS 2021)☆116Updated 2 years ago