本项目提供一个从零开始构建多模态大模型（MLLM）的完整实践路径，使用纯 PyTorch 手写 Transformer、Vision Transformer 和 GPT-style LLM，并通过 Connector 融合成可“看图说话”的多模态模型。同时，项目支持基于 SCST 的强化学习微调，实现从监督训练到奖励优化的端到端扩展。
☆36May 12, 2026Updated 5 months ago
Alternatives and similar repositories for MLLM_from_scratch_with_extension
Users that are interested in MLLM_from_scratch_with_extension are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Official pytorch repository for "Ambiguity-Restrained Text-Video Representation Learning for Partially Relevant Video Retrieval" (AAAI 20…☆11Jan 9, 2025Updated last year
- Official repo for ”Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thought“☆29Mar 29, 2026Updated 7 months ago
- [Official, NeurIPS 2025] TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs.☆32Jun 8, 2026Updated 4 months ago
- [CVPR2026] Mistake Attribution: Fine-Grained Mistake Understanding in Egocentric Videos☆16Aug 24, 2026Updated 2 months ago
- FinSmart：商业可信智能问答平台，是一款融汇商业、金融、法律等多元化知识的大语言模型。该平台以先进的InternLM2模型为基石，通过Xtuner技术的精准自蒸馏优化，大幅提升了模型的性能与准确性。同时，平台还构建了专业而系统的多个领域知识图谱，涵盖了金融领域的各个方…☆13Mar 15, 2024Updated 2 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- [NeurIPS 2024] Unsupervised Hierarchy-Agnostic Segmentation: Parsing Semantic Image Structure☆13Nov 27, 2025Updated 11 months ago
- 弱光图像增强入门指南☆29May 30, 2026Updated 5 months ago
- 基于ResNet的垃圾分类智能识别系统☆12Apr 17, 2024Updated 2 years ago
- ☆11Updated this week
- [CVPR 2026] PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewards☆10Apr 9, 2026Updated 6 months ago
- Text Proxy: Decomposing Retrieval from a 1-to-N Relationship into N 1-to-1 Relationships for Text-Video Retrieval -- AAAI2025☆22May 8, 2026Updated 5 months ago
- [CVPR'2025] Narrating the Video: Boosting Text-Video Retrieval via Comprehensive Utilization of Frame-Level Captions☆20Jan 16, 2026Updated 9 months ago
- Qwen2-VL在文旅领域的LLaMA-Factory微调案例 The case for fine-tuning Qwen2-VL in the field of historical literature and museums☆15Sep 17, 2024Updated 2 years ago
- [CVPR2026] Official repo for "OneHOI: Unifying Human-Object Interaction Generation and Editing"☆14Jul 15, 2026Updated 3 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- 国科大深度学习课程资料 2022年春季课程 王亮、黄岩教师授课☆13Jun 1, 2022Updated 4 years ago
- ☆14Oct 8, 2026Updated 3 weeks ago
- 政策新闻领域 实体识别+关系抽取 基于4000句txt微调得到☆14Aug 11, 2026Updated 2 months ago
- ☆19Jun 6, 2025Updated last year
- [AAAI 2025] Official implementation of the paper "Zero-Shot Low-Light Image Enhancement via Latent Diffusion Models"☆18Jun 14, 2025Updated last year
- ArterialNet reconstructs arterial blood pressure (ABP) waveform☆15Feb 24, 2025Updated last year
- [ICLR 2025] TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval☆27Feb 13, 2025Updated last year
- Grounded Visual Token Sampling (GroundVTS), a Vid-LLM architecture designed to enhance VTG performance through adaptive and efficient vis…☆18Jun 12, 2026Updated 4 months ago
- Nankai University Thesis LaTeX Template (南开大学硕博毕业论文模版 v2026.5)☆20May 31, 2026Updated 5 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- ☆15Aug 2, 2024Updated 2 years ago
- ASID-Caption: Attribute-Structured and Quality-Verified Audiovisual Instruction Dataset and Training Pipeline for Fine-Grained Video Unde…☆70Mar 3, 2026Updated 8 months ago
- ReTA: Reliable Test-Time Adaptation. (ACM MM 2025)☆14Aug 10, 2025Updated last year
- [CVPR'24] Solving the Catastrophic Forgetting Problem in Generalized Category Discovery https://arxiv.org/pdf/2501.05272☆17Dec 24, 2024Updated last year
- [CVPR 2024] Evaluation Toolkit of Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and R…☆10Jun 6, 2024Updated 2 years ago
- official repository of article "CrystaL: Spontaneous Emergence of Visual Latents in MLLMs"☆18May 26, 2026Updated 5 months ago
- 论文绘图-局部细节放大☆22May 21, 2025Updated last year
- EEG-based Emotion Recognition using Deep Reinforcement Learning☆17Dec 31, 2023Updated 2 years ago
- QuantClass（邢不行量化课）数据增量同步工具，macOS，CLI + GUI☆16Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- BTC 实时交易信号系统，结合订单流、多因子共振分析与机器学习模型。BTC real-time trading signal system, combining order flow, multi-factor resonance analysis and machine …☆12Feb 22, 2026Updated 8 months ago
- (ICCV 25) From Enhancement to Understanding: Build a Generalized Bridge for Low-light Vision via Semantically Consistent Unsupervised Fin…☆20Jul 21, 2025Updated last year
- ☆17May 14, 2026Updated 5 months ago
- ☆11May 8, 2026Updated 5 months ago
- Continuous Exposure Learning for Low-light Image Enhancement using Neural ODEs, in ICLR 2025 (Spotlight)☆21Mar 29, 2025Updated last year
- 聚宽量化策略集合 - ETF轮动、股票选股、跨市场套利等策略☆16Mar 27, 2026Updated 7 months ago
- pytorch implementation of DeepSeek Engram☆19Mar 24, 2026Updated 7 months ago