本项目提供一个从零开始构建多模态大模型（MLLM）的完整实践路径，使用纯 PyTorch 手写 Transformer、Vision Transformer 和 GPT-style LLM，并通过 Connector 融合成可“看图说话”的多模态模型。同时，项目支持基于 SCST 的强化学习微调，实现从监督训练到奖励优化的端到端扩展。
☆28May 12, 2026Updated 2 months ago
Alternatives and similar repositories for MLLM_from_scratch_with_extension
Users that are interested in MLLM_from_scratch_with_extension are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- [Official, NeurIPS 2025] TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs.☆17Jun 8, 2026Updated last month
- Official repo for ”Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thought“☆26Mar 29, 2026Updated 3 months ago
- Implementation of Motion Artifacts on wearable PPG signals for ambulatory heart rate monitoring☆12Nov 21, 2022Updated 3 years ago
- [MICCAI-STACOM 2022] This repository is the official implementation of "Learning correspondences of cardiac motion from images using biom…☆13Sep 30, 2022Updated 3 years ago
- Qwen2-VL在文旅领域的LLaMA-Factory微调案例 The case for fine-tuning Qwen2-VL in the field of historical literature and museums☆15Sep 17, 2024Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆19Jun 6, 2025Updated last year
- Nankai University Thesis LaTeX Template (南开大学硕博毕业论文模版 v2026.5)☆17May 31, 2026Updated last month
- 基于 Java、SQLite 与 MPAndroidChart 的 Android 积分任务管理应用。☆13Updated this week
- 国科大深度学习课程资料 2022年春季课程 王亮、黄岩教师授课☆13Jun 1, 2022Updated 4 years ago
- ASID-Caption: Attribute-Structured and Quality-Verified Audiovisual Instruction Dataset and Training Pipeline for Fine-Grained Video Unde…☆67Mar 3, 2026Updated 4 months ago
- 新手友好的基于 Qwen2 的生成式推荐系统，通过大模型理解用户偏好生成候选物品，融合 TF-IDF 关键词召回、热门物品召回构建多源策略，平衡相关性与多样性。采用 LightGBM 排序模型精准打分，内置召回率、NDCG 等评估指标量化效果。通过 Flask 封装 API…☆53Dec 6, 2025Updated 7 months ago
- OPARL(Optimistic and Pessimistic Actor in RL)☆18Jan 26, 2024Updated 2 years ago
- Official Repo for paper "VLCache: Computing 2% Vision Tokens and Reusing 98% for Vision-Language Inference"☆16Mar 28, 2026Updated 3 months ago
- - 【LLM面经】大模型实习面试指南。手撕代码、面经经验、思考题等。初学者 学习ing......欢迎指正错误☆35Nov 11, 2025Updated 8 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- 面向 AI Agent 入门者的教学项目。通过一个完整的电商客服 Agent，让你亲眼看到每一轮对话中 Agent 内部经历了哪些环节、做了什么决策、 调用了什么工具、生成了什么 Prompt。☆48Mar 15, 2026Updated 4 months ago
- Global 3D Map Construction using RTAB-Map iPhone/iPad app data☆13Jun 15, 2021Updated 5 years ago
- CBLUE2.0-关系抽取模型，基于pytorch☆18Oct 23, 2024Updated last year
- Replication of "Taming the Factor Zoo: A Test of New Factors (Feng, Giglio, and Xiu, 2020, JF)"☆10Mar 4, 2024Updated 2 years ago
- Verilog写的简单五级流水线CPU☆16Sep 24, 2020Updated 5 years ago
- ☆24Feb 16, 2022Updated 4 years ago
- 安卓课设 五邑大学课程表app Android 课程设计☆19Jun 11, 2023Updated 3 years ago
- This repositories contains the reference implementation for the Sparse Delta Memory paper.More precisely, it contains the model definitio…☆30Jul 9, 2026Updated last week
- This code is written to participate in the MRBrainS18 competition.☆20Jan 17, 2022Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Husky-LIO-SAM☆12Feb 23, 2023Updated 3 years ago
- 天池新闻推荐项目☆68May 17, 2025Updated last year
- ☆11Jun 16, 2020Updated 6 years ago
- Android Music Streaming App with firebase☆21Feb 19, 2024Updated 2 years ago
- 录制Android手机的图像和IMU数据，标定并运行VINS-Mono☆11Mar 4, 2022Updated 4 years ago
- ☆14Feb 28, 2019Updated 7 years ago
- ☆14Jan 31, 2024Updated 2 years ago
- 基于资金流的择时选股策略☆18May 25, 2018Updated 8 years ago
- 丁立中的大模型算法工程作品集。聚焦 LLM / VLM 全链路的复现与优化，涵盖：① 预训练与微调（Pretrain / SFT / MoE / 多模态对齐）；② 强化学习对齐（PPO / GRPO / DAPO，含多奖励函数与 GAE）；③ 知识蒸馏（离线 KL 蒸馏、在…☆17May 26, 2026Updated last month
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Cross-city Semantic Segmentation (C2Seg) in Multimodal Remote Sensing: Outcome of the 2023 IEEE WHISPERS C2Seg Challenge☆24May 10, 2024Updated 2 years ago
- verify OrderBook Tick Data Trading Strategy on futures.☆17Mar 21, 2019Updated 7 years ago
- 通过icp解决evo分析轨迹时，SLAM输出的轨迹和gt轨迹无法对齐的问题☆14Apr 3, 2024Updated 2 years ago
- 机构级 A股量化系统 - Hermes 多智能体 + Barra 中性化 + Level2 微结构 + 15 个圈内 tricks 完整实现☆15Updated this week
- MVS-VIO: Dense Monocular Visual Inertial Odometry with Lightweight MVSNET☆13Apr 21, 2023Updated 3 years ago
- Codes and data for paper "Rigid and Non-rigid Motion Artifact Reduction in X-ray CT using Attention Module"☆27Aug 12, 2022Updated 3 years ago
- This GitHub repo features a project on monocular depth estimation for 3D mapping and path planning in partially known environments using …☆12Apr 25, 2023Updated 3 years ago