FastTrack4LLM 是一个为大模型学习者准备的大模型学习与实践框架，帮助他们轻松掌握大模型的核心原理与训练流程，让每个人都能真正理解大模型的内部机制。本项目不仅完整复现了 LLaMA、Qwen、DeepSeek 等主流开源大模型架构，还覆盖了大模型的全生命周期：Tokenizer 训练、预训练、全量微调、参数高效微调（LoRA）、人类反馈对齐（DPO）、知识蒸馏等。不同于仅仅调用API或使用现成模型，我们带你从零开始，亲手构建、训练、优化属于自己的大语言模型，快来体验吧！
☆31Nov 6, 2025Updated 8 months ago
Alternatives and similar repositories for FastTrack4LLM
Users that are interested in FastTrack4LLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- NewSP: A New Search Process for Continuous Subgraph Matching over Dynamic Graphs[ICDE 24]☆16Oct 10, 2024Updated last year
- 本项目从零开始构建并优化了一个千万参数级别的大规模预训练语言模型，涵盖预训练、有监督微调（SFT）和R1推理蒸馏三个阶段。项目采用自定义Transformer架构（包括RMSNorm、分组注意力、多Query机制、SwiGLU激活和RoPE位置编码），实现高效的长文本处理和…☆23Mar 10, 2025Updated last year
- 本项目旨在系统性学习和记录大语言模型（LLM）系统领域的核心知识，重点关注分布式训练、推理优化、强化学习（RLHF）对齐的原理、主流框架和工程实践。☆15Apr 9, 2026Updated 3 months ago
- 丁立中的大模型算法工程作品集。聚焦 LLM / VLM 全链路的复现与优化，涵盖：① 预训练与微调（Pretrain / SFT / MoE / 多模态对齐）；② 强化学习对齐（PPO / GRPO / DAPO，含多奖励函数与 GAE）；③ 知识蒸馏（离线 KL 蒸馏、在…☆17May 26, 2026Updated 2 months ago
- A library for detecting hallucination and improving LLM factuality☆20Sep 14, 2025Updated 10 months ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- ☆16Apr 2, 2024Updated 2 years ago
- Develop a python application that allows you to extract valuable insights, engage in meaningful conversations, and explore video content …☆13Jan 24, 2024Updated 2 years ago
- A sophisticated web application designed to revolutionize the resume screening process by harnessing the power of multiple state-of-the-a…☆11Mar 13, 2025Updated last year
- 本项目基于 HuggingFace Transformers 和 PEFT (LoRA)，专注于中文法律文本的因果语言模型微调。首先，利用 Qwen2-70B 大模型对下载的法律裁判文书进行思维链（Chain-of-Thought, CoT）抽取，实现模型蒸馏与知识迁移；随…☆19Aug 10, 2025Updated 11 months ago
- 在千问最新的多模态image-text模型Qwen3-VL-4B-Instruct 进行多种lora微调对比效果，通过langchain+RAG+多智能体(Multi-Agent)进行部署☆55Dec 14, 2025Updated 7 months ago
- How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients☆20Jun 17, 2025Updated last year
- Code for Analyzing Redundancy in Pretrained Transformer Models accepted at EMNLP 2020☆14Oct 6, 2020Updated 5 years ago
- CoCoFL: Communication- and Computation-Aware Federated Learning via Partial NN Freezing and Quantization☆13Aug 3, 2024Updated last year
- Developing a high-precision legal expert LLM application called Contract Advisor RAG. The project's goal is to create a Retrieval Augment…☆15Apr 10, 2024Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- (ICCV 2023) Official implementation of Rectified Straight Through Estimator (ReSTE).☆34Sep 20, 2024Updated last year
- [ICASSP'2025] "M³Rec: Selective State Space Models with Mixture-of-Modality Experts for Multi-Modal Sequential Recommendation"☆14Jul 9, 2025Updated last year
- Huggingface PPO Demo☆30Sep 7, 2025Updated 10 months ago
- ☆18Aug 17, 2014Updated 11 years ago
- [CVPR2023] Practical Network Acceleration with Tiny Sets☆13Jul 28, 2023Updated 2 years ago
- Source Code for "Joint Entity and Relation Extraction with Span Pruning and Hypergraph Neural Networks"☆33May 29, 2026Updated last month
- [NeurIPS2022] Where to Pay Attention in Sparse Training for Feature Selection?☆12Feb 10, 2023Updated 3 years ago
- VHDL Implementation☆15Oct 9, 2014Updated 11 years ago
- [ICML2024] "FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees" by Jiaha…☆14Sep 22, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆29May 5, 2026Updated 2 months ago
- Q-RR, DIANA-RR, Q-NASTYA, NASTYA-DIANA, QSGD, DIANA, FedCOM and FedPAQ on logistic loss with L2 regularization☆11Nov 1, 2022Updated 3 years ago
- [Findings of EMNLP22] From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models☆19Mar 16, 2023Updated 3 years ago
- The official Github repository for paper "R^2AG: Incorporating Retrieval Information into Retrieval Augmented Generation" (EMNLP 2024 Fin…☆41Dec 6, 2024Updated last year
- Dummy form filler for Firefox & Chrome☆20Dec 4, 2025Updated 7 months ago
- Fire Together Wire Together: A Dynamic Pruning Approach with Self-Supervised Mask Prediction☆10May 25, 2022Updated 4 years ago
- ☆12Dec 26, 2024Updated last year
- Pyhon code for my Towards Data Science articles☆13Jan 19, 2024Updated 2 years ago
- [AAAI 2023 Oral] Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training☆14Apr 19, 2023Updated 3 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- 用XGBoost算法训练信用卡反欺诈预测模型。数据集太大（175MB）超过限制，未能上传，可查询。☆11Mar 13, 2020Updated 6 years ago
- This repository contains the official implementation of the paper entitled with "FedAPEN: Personalized Cross-silo Federated Learning with…☆14Dec 4, 2023Updated 2 years ago
- The official implementation of "Federated Learning with Label-Masking Distillation"☆11Oct 28, 2023Updated 2 years ago
- Device configuration for HTC VIVOW☆15Oct 7, 2014Updated 11 years ago
- Official implementation of DapperFL.☆13Oct 29, 2024Updated last year
- Code for Adaptive Deep Neural Network Inference Optimization with EENet☆13Mar 28, 2024Updated 2 years ago
- 基于Qwen2+SFT+DPO的医疗问答系统，项目中使用了自定义的 SFTTrainer/DPOTrainer/TRPOTrainer用于训练，其次，项目还调用各种知识库工具（neo4j, milvus, LDA, 等）进行自动化训练数据生成。另外，使用 vllm 用于推理…☆89Apr 29, 2026Updated 2 months ago