FastTrack4LLM 是一个为大模型学习者准备的大模型学习与实践框架，帮助他们轻松掌握大模型的核心原理与训练流程，让每个人都能真正理解大模型的内部机制。本项目不仅完整复现了 LLaMA、Qwen、DeepSeek 等主流开源大模型架构，还覆盖了大模型的全生命周期：Tokenizer 训练、预训练、全量微调、参数高效微调（LoRA）、人类反馈对齐（DPO）、知识蒸馏等。不同于仅仅调用API或使用现成模型，我们带你从零开始，亲手构建、训练、优化属于自己的大语言模型，快来体验吧！
☆31Nov 6, 2025Updated 11 months ago
Alternatives and similar repositories for FastTrack4LLM
Users that are interested in FastTrack4LLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 本项目从零开始构建并优化了一个千万参数级别的大规模预训练语言模型，涵盖预训练、有监督微调（SFT）和R1推理蒸馏三个阶段。项目采用自定义Transformer架构（包括RMSNorm、分组注意力、多Query机制、SwiGLU激活和RoPE位置编码），实现高效的长文本处理和…☆24Mar 10, 2025Updated last year
- This repository contains the code and resources for our ACL 2024 Findings paper, "Deciphering Digital Detectives: Understanding LLM Behav…☆10Dec 9, 2025Updated 10 months ago
- Exploiting Forgery-Induced Alignment Bias for Generalisable AI-Generated Image Detection☆14Dec 8, 2025Updated 10 months ago
- Official implementation of NeurIPS 2025 paper "Visual Instruction Bottleneck Tuning"☆15Apr 10, 2026Updated 6 months ago
- EMNLP 2022 paper: Boosting Document-Level Relation Extraction by Mining and Injecting Logical Rules☆12Jun 26, 2023Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆11Jul 19, 2024Updated 2 years ago
- Official implementation of ICML 2025 paper "Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach"☆12May 27, 2025Updated last year
- 2025.01：从零到一实现了一个多模态大模型，并命名为Reyes（睿视），R：睿，eyes：眼。Reyes的参数量为8B，视觉编码器使用的是InternViT-300M-448px-V2_5,语言模型侧使用的是Qwen2.5-7B-Instruct，Reyes也通过一个两…☆34Feb 10, 2026Updated 8 months ago
- Interactive SQL generation via editable step-by-step explanation☆18Sep 12, 2024Updated 2 years ago
- Conditional Mutual Informaation Neural Estimator☆15Oct 23, 2020Updated 6 years ago
- This project is a versatile and powerful search tool that leverages state-of-the-art natural language processing models to provide releva…☆12Apr 3, 2023Updated 3 years ago
- An app specifically designed to track issues.☆14May 23, 2023Updated 3 years ago
- ☆19Apr 14, 2026Updated 6 months ago
- Embodied-OS：面向具身智能的分层运行时——Driver·Capability·Skill·Brain 全栈参考实现☆13Sep 25, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- [ICDCS 2023] Evaluation and Optimization of Gradient Compression for Distributed Deep Learning☆10Apr 28, 2023Updated 3 years ago
- ☆12Mar 1, 2023Updated 3 years ago
- CoCoFL: Communication- and Computation-Aware Federated Learning via Partial NN Freezing and Quantization☆12Aug 3, 2024Updated 2 years ago
- 在千问最新的多模态image-text模型Qwen3-VL-4B-Instruct 进行多种lora微调对比效果，通过langchain+RAG+多智能体(Multi-Agent)进行部署☆60Dec 14, 2025Updated 10 months ago
- ☆10Oct 16, 2024Updated 2 years ago
- Code for Analyzing Redundancy in Pretrained Transformer Models accepted at EMNLP 2020☆14Oct 6, 2020Updated 6 years ago
- 使用ptuningv2，对比学习，训练prompt bert，增强句向量表征能力☆11Jun 15, 2023Updated 3 years ago
- Browse, search, and manage Vinted listings from Claude or the terminal — buy, sell, track items, and automate your secondhand workflow☆19Oct 7, 2026Updated 3 weeks ago
- [ICML 2022] This work investigates the compatibility between label smoothing (LS) and knowledge distillation (KD). We suggest to use an L…☆13Dec 8, 2022Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- VHDL Implementation☆15Oct 9, 2014Updated 12 years ago
- 基于Multi-Agent架构的智能旅游助手☆29Mar 24, 2026Updated 7 months ago
- 基于ReAct推理框架的企业知识库Agent | 多跳推理 + 智能复杂度判断 + 自动工作流编排 + 反思评估☆12Jul 12, 2025Updated last year
- Fine-tuning, DPO, RLHF, RLAIF on LLMs - Qwen3, Zephyr 7B GPTQ with 4-Bit Quantization, Mistral-7B-GPTQ☆15Jul 5, 2025Updated last year
- 基于 Qwen2-0.5B 以及 SigLIP 实现的轻量化多模态风格化问答大模型☆32Aug 8, 2025Updated last year
- Federated Model Heterogeneous Matryoshka Representation Learning (NeurIPS'24)☆13Oct 30, 2024Updated 2 years ago
- This codebase contains the python scripts for the model for the "Learning Co-Speech Gesture for Multimodal Aphasia Type Detection (EMNLP …☆10Dec 19, 2023Updated 2 years ago
- Huggingface PPO Demo☆31Sep 7, 2025Updated last year
- [NeuIPS'24] FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction☆14Dec 3, 2024Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Advanced LLM fine-tuning techniques: SFT (LoRA, QLoRA, DoRA, P-/Prefix-Tuning), GRPO, DPO, ORPO, KTO & PPO; composable correctness/format…☆14Updated this week
- (CVPR2025) The implementation of the paper 'Federated Learning with Domain Shift Eraser'☆14Jan 14, 2026Updated 9 months ago
- [ICLR'24] Heterogeneous Personalized Federated Learning by Local-Global Updates Mixing via Convergence Rate☆14Jun 17, 2025Updated last year
- Pyhon code for my Towards Data Science articles☆13Jan 19, 2024Updated 2 years ago
- The code repository for "RCNet: Reverse Feature Pyramid and Cross-scale Shift Network for Object Detection" (ACM MM'21)☆11Oct 23, 2021Updated 5 years ago
- 🦞 OpenClaw AI Agent Scenario Workshop — visually install, configure & manage OpenClaw agents in 5 minutes. No terminal needed.☆16Mar 14, 2026Updated 7 months ago
- 本项目对Deepseek-R1-Distill-Qwen-7B进行心理咨询CoT数据的LoRA微调，以进一步提升Deepseek-R1-Distill-Qwen-7B在心理咨询领域的慢思考能力。☆12Mar 11, 2025Updated last year