本项目提供一个从零开始构建多模态大模型（MLLM）的完整实践路径，使用纯 PyTorch 手写 Transformer、Vision Transformer 和 GPT-style LLM，并通过 Connector 融合成可“看图说话”的多模态模型。同时，项目支持基于 SCST 的强化学习微调，实现从监督训练到奖励优化的端到端扩展。
☆31May 12, 2026Updated 3 months ago
Alternatives and similar repositories for MLLM_from_scratch_with_extension
Users that are interested in MLLM_from_scratch_with_extension are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆10Nov 27, 2024Updated last year
- [𝗜𝗖𝗠𝗟 𝟮𝟬𝟮𝟲] Dispersion loss counteracts embedding condensation and improves generalization in small language models☆19May 21, 2026Updated 3 months ago
- [MICCAI-STACOM 2022] This repository is the official implementation of "Learning correspondences of cardiac motion from images using biom…☆13Sep 30, 2022Updated 3 years ago
- ☆14Aug 14, 2022Updated 4 years ago
- [ACMMM 2026] PLUME: Latent Reasoning Based Universal Multimodal Embedding☆25Apr 29, 2026Updated 4 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ASID-Caption: Attribute-Structured and Quality-Verified Audiovisual Instruction Dataset and Training Pipeline for Fine-Grained Video Unde…☆69Mar 3, 2026Updated 5 months ago
- Time Series Forecasting with Dynamic Graph Modeling☆16Aug 31, 2025Updated last year
- [ICLR 2025] TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval☆27Feb 13, 2025Updated last year
- Accelerated motion correction with score-based generative models☆20Sep 8, 2024Updated last year
- 新手友好的基于 Qwen2 的生成式推荐系统，通过大模型理解用户偏好生成候选物品，融合 TF-IDF 关键词召回、热门物品召回构建多源策略，平衡相关性与多样性。采用 LightGBM 排序模型精准打分，内置召回率、NDCG 等评估指标量化效果。通过 Flask 封装 API…☆54Dec 6, 2025Updated 8 months ago
- Code for Fast-weight Product Key Memory (FwPKM)☆22Mar 18, 2026Updated 5 months ago
- The Pytorch implemetation of "FeatWalk: Enhancing Few-Shot Classification through Local View Leveraging", AAAI 2024.☆11Mar 4, 2024Updated 2 years ago
- - 【LLM面经】大模型实习面试指南。手撕代码、面经经验、思考题等。初学者学习ing......欢迎指正错误☆35Nov 11, 2025Updated 9 months ago
- Global 3D Map Construction using RTAB-Map iPhone/iPad app data☆13Jun 15, 2021Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Replication of "Taming the Factor Zoo: A Test of New Factors (Feng, Giglio, and Xiu, 2020, JF)"☆10Mar 4, 2024Updated 2 years ago
- Implementation for Residual Registration Network (R2Net)☆15Aug 4, 2023Updated 3 years ago
- ☆12Jul 19, 2020Updated 6 years ago
- ☆24Feb 16, 2022Updated 4 years ago
- How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients☆21Jun 17, 2025Updated last year
- This repositories contains the reference implementation for the Sparse Delta Memory paper.More precisely, it contains the model definitio…☆37Jul 9, 2026Updated last month
- Husky-LIO-SAM☆12Feb 23, 2023Updated 3 years ago
- This code is written to participate in the MRBrainS18 competition.☆20Jan 17, 2022Updated 4 years ago
- ☆11Jun 16, 2020Updated 6 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- CNN to detect Pneumonia using Chest X-Rays☆21Dec 8, 2022Updated 3 years ago
- 历年EMNLP论文和开源项目合集，包含EMNLP2025、EMNLP2024、EMNLP2023、EMNLP2022、EMNLP2021。☆18Jul 16, 2025Updated last year
- [STACOM-MICCAI 2019] Deep Learning Registration for Cardiac Motion Tracking☆17Jan 30, 2024Updated 2 years ago
- Learns effective selective labeling strategies for medical images using deep reinforcement learning and meta learning☆27May 8, 2021Updated 5 years ago
- 录制Android手机的图像和IMU数据，标定并运行VINS-Mono☆11Mar 4, 2022Updated 4 years ago
- ☆14Feb 28, 2019Updated 7 years ago
- ☆15Jan 31, 2024Updated 2 years ago
- 基于资金流的择时选股策略☆18May 25, 2018Updated 8 years ago
- 丁立中的大模型算法工程作品集。聚焦 LLM / VLM 全链路的复现与优化，涵盖：① 预训练与微调（Pretrain / SFT / MoE / 多模态对齐）；② 强化学习对齐（PPO / GRPO / DAPO，含多奖励函数与 GAE）；③ 知识蒸馏（离线 KL 蒸馏、在…☆17May 26, 2026Updated 3 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- verify OrderBook Tick Data Trading Strategy on futures.☆17Mar 21, 2019Updated 7 years ago
- 通过icp解决evo分析轨迹时，SLAM输出的轨迹和gt轨迹无法对齐的问题☆14Apr 3, 2024Updated 2 years ago
- [ICML2026] Reproduce Kimi K1.5/K2 RL algorithm and rollout system☆21Apr 9, 2026Updated 4 months ago
- 机构级 A股量化系统 - Hermes 多智能体 + Barra 中性化 + Level2 微结构 + 15 个圈内 tricks 完整实现☆21Aug 4, 2026Updated 3 weeks ago
- MVS-VIO: Dense Monocular Visual Inertial Odometry with Lightweight MVSNET☆13Apr 21, 2023Updated 3 years ago
- BroadBEV: Collaborative LiDAR-camera Fusion for Broad-sighted Bird's Eye View Map Construction, in 2024 ICRA.☆14May 2, 2024Updated 2 years ago
- 天津大学幻灯片（PPT or Slide）模板，由LaTeX Beamer制作，基于Berlin主题，可用于毕业设计答辩、项目答辩、小组展示等多方面，允许输出优美的数学公式和算法代码。☆22May 29, 2022Updated 4 years ago