sxontheway / Keep-Learning
The record of what I‘ve been through.
☆95Updated this week
Related projects: ⓘ
- An awesome gpu tasks scheduler. 轻量好用的GPU机群任务调度工具。觉得有用可以点个star☆159Updated 2 years ago
- pytorch单精度、半精度、混合精度、单卡、多卡（DP / DDP）、FSDP、DeepSpeed模型训练代码，并对比不同方法的训练速度以及GPU内存的使用☆69Updated 6 months ago
- DeepSpeed教程 & 示例注释 & 学习笔记 （大模型高效训练）☆94Updated last year
- ☆125Updated last week
- ATC23 AE☆42Updated last year
- 🌱 梦想家(DreamerGPT)：中文大语言模型指令精调☆48Updated last year
- Must-read Papers of Parameter-Efficient Tuning (Delta Tuning) Methods on Pre-trained Models.☆272Updated last year
- 欢迎来到 "LLM-travel" 仓库！探索大语言模型（LLM）的奥秘 🚀。致力于深入理解、探讨以及实现与大模型相关的各种技术、原理和应用。☆246Updated 2 months ago
- Cool Papers - Immersive Paper Discovery☆352Updated last week
- DeepSpeed Tutorial☆80Updated last month
- A light-weight script for maintaining a LOT of machine learning experiments.☆88Updated last year
- LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training☆389Updated this week
- The Roadmap for LLMs☆84Updated last year
- A MoE impl for PyTorch, [ATC'23] SmartMoE☆56Updated last year
- ☆70Updated 9 months ago
- A collection of phenomenons observed during the scaling of big foundation models, which may be developed into consensus, principles, or l…☆274Updated last year
- Models and examples built with OneFlow☆94Updated last month
- 基于Gated Attention Unit的Transformer模型（尝鲜版）☆95Updated last year
- an implementation of transformer, bert, gpt, and diffusion models for learning purposes☆139Updated 10 months ago
- Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.☆90Updated 7 months ago
- Survey Paper List - Efficient LLM and Foundation Models☆190Updated 6 months ago
- Code for "Lion: Adversarial Distillation of Proprietary Large Language Models (EMNLP 2023)"☆195Updated 7 months ago
- ☆77Updated 2 months ago
- LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment☆191Updated 4 months ago
- ☆49Updated last year
- An automated pipeline for evaluating LLMs for role-playing.☆118Updated last week
- ☆82Updated last year
- [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408☆188Updated last year
- Paper List for In-context Learning 🌷☆165Updated 7 months ago
- 主要记录大语言大模型（LLMs） 算法（应用）工程师多模态相关知识☆62Updated 4 months ago