preacher-1 / MLA_tutorial
from MHA, MQA, GQA to MLA by 苏剑林, with code
☆13Updated last month
Alternatives and similar repositories for MLA_tutorial:
Users that are interested in MLA_tutorial are comparing it to the libraries listed below
- ThinkLLM:大语言模型算法与组件实现☆26Updated this week
- 大语言模型应用:RAG、NL2SQL、聊天机器人、预训练、MOE混合专家模型、微调训练、强化学习、天池数据竞赛☆58Updated last month
- LLM101n: Let's build a Storyteller 中文版☆130Updated 7 months ago
- ☆72Updated last month
- 通义千问的DPO训练☆40Updated 6 months ago
- 使用单个24G显卡,从0开始训练LLM☆50Updated 5 months ago
- unify-easy-llm(ULM)旨在打造一个简易的一键式大模型训练工具,支持Nvidia GPU、Ascend NPU等不同硬件以及常用的大模型。☆55Updated 8 months ago
- ☆102Updated last week
- an implementation of transformer, bert, gpt, and diffusion models for learning purposes☆152Updated 5 months ago
- ☆105Updated 4 months ago
- 从零到一实现一个 miniLLM~(动手学习LLM)☆64Updated 10 months ago
- 怎么训练一个LLM分词器☆142Updated last year
- OpenSeek aims to unite the global open source community to drive collaborative innovation in algorithms, data and systems to develop next…☆127Updated this week
- 通过动画学强化学习笔记☆38Updated last month
- 大模型/LLM推理和部署理论与实践☆221Updated 3 weeks ago
- LLM Tokenizer with BPE algorithm☆31Updated 10 months ago
- DeepSeek Native Sparse Attention pytorch implementation☆46Updated 3 weeks ago
- nlp_interview notes and answers: 该仓库主要记录 NLP 算法工程师相关的面试题和参考答案☆20Updated last year
- Inference code for LLaMA models☆118Updated last year
- 天池算法比赛《BetterMixture - 大模型数据混合挑战赛》的第一名top1解决方案☆27Updated 8 months ago
- ☆50Updated 6 months ago
- 解锁HuggingFace生态的百般用法☆88Updated 3 months ago
- pytorch分布式训练☆64Updated last year
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆56Updated 11 months ago
- WWW2025 Multimodal Intent Recognition for Dialogue Systems Challenge☆118Updated 4 months ago
- LLM 推理服务性能测试☆39Updated last year
- A visuailzation tool to make deep understaning and easier debugging for RLHF training.☆177Updated last month
- ☆29Updated 7 months ago
- the newest version of llama3,source code explained line by line using Chinese☆22Updated 11 months ago
- ☆22Updated last month