sxontheway / Keep-Learning
The record of what I‘ve been through.
☆95Updated 2 months ago
Alternatives and similar repositories for Keep-Learning:
Users that are interested in Keep-Learning are comparing it to the libraries listed below
- Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.☆94Updated last year
- ☆78Updated last year
- ☆52Updated last year
- pytorch单精度、半精度、混合精度、单卡、多卡（DP / DDP）、FSDP、DeepSpeed模型训练代码，并对比不同方法的训练速度以及GPU内存的使用☆92Updated last year
- ☆84Updated last year
- DeepSpeed教程 & 示例注释 & 学习笔记 （大模型高效训练）☆151Updated last year
- ATC23 AE☆45Updated last year
- LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training☆397Updated 2 months ago
- A MoE impl for PyTorch, [ATC'23] SmartMoE☆61Updated last year
- Models and examples built with OneFlow☆96Updated 5 months ago
- A collection of phenomenons observed during the scaling of big foundation models, which may be developed into consensus, principles, or l…☆277Updated last year
- An awesome gpu tasks scheduler. 轻量好用的GPU机群任务调度工具。觉得有用可以点个star☆177Updated 2 years ago
- an implementation of transformer, bert, gpt, and diffusion models for learning purposes☆151Updated 5 months ago
- A high-performance distributed deep learning system targeting large-scale and automated distributed training. If you have any interests, …☆108Updated last year
- A light-weight script for maintaining a LOT of machine learning experiments.☆91Updated 2 years ago
- 欢迎来到 "LLM-travel" 仓库！探索大语言模型（LLM）的奥秘 🚀。致力于深入理解、探讨以及实现与大模型相关的各种技术、原理和应用。☆303Updated 8 months ago
- Collaborative Training of Large Language Models in an Efficient Way☆413Updated 6 months ago
- Finetuning LLaMA with RLHF (Reinforcement Learning with Human Feedback) based on DeepSpeed Chat☆114Updated last year
- Official Repository for SIGIR2024 Demo Paper "An Integrated Data Processing Framework for Pretraining Foundation Models"☆73Updated 6 months ago
- Converting Mixtral-8x7B to Mixtral-[1~7]x7B☆22Updated last year
- LLM Inference with Deep Learning Accelerator.☆33Updated 2 months ago
- The blog, read report and code example for AGI/LLM related knowledge.☆36Updated last month
- Rectified Rotary Position Embeddings☆360Updated 10 months ago
- ☆182Updated 5 months ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆69Updated last year
- ☆105Updated 4 months ago
- Pretrain、decay、SFT a CodeLLM from scratch 🧙♂️☆35Updated 10 months ago
- ☆125Updated 3 weeks ago
- Transformer related optimization, including BERT, GPT☆59Updated last year
- A flexible and efficient training framework for large-scale alignment tasks☆333Updated last month