Oneflow-Inc / libai
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
☆397Updated 2 months ago
Alternatives and similar repositories for libai:
Users that are interested in libai are comparing it to the libraries listed below
- Models and examples built with OneFlow☆96Updated 3 months ago
- ☆211Updated last year
- Efficient Training (including pre-training and fine-tuning) for Big Models☆574Updated 5 months ago
- FlagScale is a large model toolkit based on open-sourced projects.☆207Updated this week
- optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052☆467Updated 10 months ago
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆400Updated 2 weeks ago
- Best practice for training LLaMA models in Megatron-LM☆638Updated last year
- Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.☆266Updated last year
- ☆302Updated 3 weeks ago
- ☆76Updated last year
- Model Compression for Big Models☆152Updated last year
- Efficient Inference for Big Models☆574Updated last year
- Running BERT without Padding☆468Updated 2 years ago
- RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.☆582Updated 3 months ago
- DeepLearning Framework Performance Profiling Toolkit☆281Updated 2 years ago
- InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencie…☆326Updated this week
- ☆598Updated 7 months ago
- OneFlow models for benchmarking.☆105Updated 5 months ago
- The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.☆798Updated this week
- Collaborative Training of Large Language Models in an Efficient Way☆411Updated 4 months ago
- ☆151Updated last month
- A flexible and efficient training framework for large-scale alignment tasks☆272Updated this week
- 更纯粹、更高压缩率的Tokenizer☆468Updated last month
- ☆127Updated 3 weeks ago
- Zero Bubble Pipeline Parallelism☆309Updated 2 months ago
- Tutel MoE: An Optimized Mixture-of-Experts Implementation☆746Updated this week
- Efficient, Low-Resource, Distributed transformer implementation based on BMTrain☆244Updated last year
- A collection of memory efficient attention operators implemented in the Triton language.☆229Updated 7 months ago
- ☆84Updated last year
- ☆317Updated 6 months ago