bobo0810 / LearnDeepSpeed
DeepSpeed教程 & 示例注释 & 学习笔记 (大模型高效训练)
☆147Updated last year
Alternatives and similar repositories for LearnDeepSpeed:
Users that are interested in LearnDeepSpeed are comparing it to the libraries listed below
- 主要记录大语言大模型(LLMs) 算法(应用)工程师多模态相关知识☆118Updated 8 months ago
- 欢迎来到 "LLM-travel" 仓库!探索大语言模型(LLM)的奥秘 🚀。致力于深入理解、探讨以及实现与大模型相关的各种技术、原理和应用。☆287Updated 6 months ago
- ☆258Updated 10 months ago
- Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conv…☆395Updated last month
- pytorch单精度、半精度、混合精度、单卡、多卡(DP / DDP)、FSDP、DeepSpeed模型训练代码,并对比不同方法的训练速度以及GPU内存的使用☆87Updated 10 months ago
- Efficient Multimodal Large Language Models: A Survey☆307Updated 5 months ago
- DeepSpeed Tutorial☆94Updated 5 months ago
- ☆96Updated 2 months ago
- LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment☆273Updated 9 months ago
- 训练一个对中文支持更好的LLaVA模型,并开源训练代码和数据。☆44Updated 4 months ago
- 包含程序员面试大厂面试题和面试经验☆116Updated last month
- WWW2025 Multimodal Intent Recognition for Dialogue Systems Challenge☆113Updated 2 months ago
- an implementation of transformer, bert, gpt, and diffusion models for learning purposes☆150Updated 3 months ago
- pytorch distribute tutorials☆101Updated 3 months ago
- Code for a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models☆57Updated 7 months ago
- ☆171Updated 3 months ago
- Reading notes about Multimodal Large Language Models, Large Language Models, and Diffusion Models☆238Updated 3 weeks ago
- Train a 1B LLM with 1T tokens from scratch by personal☆483Updated this week
- ☆52Updated last year
- 对llava官方代码的一些学习笔记☆11Updated 3 months ago
- 欢迎来到 LLM-Dojo,这里是一个开源大模型学习场所,使用简洁且易阅读的代码构建模型训练框架(支持各种主流模型如Qwen、Llama、GLM等等)、RLHF框架(DPO/CPO/KTO/PPO)等各种功能。👩🎓👨🎓☆470Updated 2 weeks ago
- Inference code for LLaMA models☆113Updated last year
- An automated pipeline for evaluating LLMs for role-playing.☆156Updated 4 months ago
- ☆102Updated 10 months ago
- TinyRAG☆254Updated 3 months ago
- This is a repository used by individuals to experiment and reproduce the pre-training process of LLM.☆386Updated 9 months ago
- 从0开始,将chatgpt的技术路线跑一遍。☆189Updated 4 months ago
- 本仓库是关于大模型面试中常见面试试题和面试经验的整理。这里收集了各类与大模型相关的面试题目,并提供详细的解答和分析。本仓库由上海交大交影社区维护☆65Updated 5 months ago
- LLM101n: Let's build a Storyteller 中文版☆122Updated 5 months ago