A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.
☆365Feb 28, 2026Updated last week
Alternatives and similar repositories for lmms-finetune
Users that are interested in lmms-finetune are comparing it to the libraries listed below
Sorting:
- A Framework of Small-scale Large Multimodal Models☆963Feb 7, 2026Updated last month
- ☆4,582Sep 14, 2025Updated 5 months ago
- 【NeurIPS 2024】Dense Connector for MLLMs☆181Oct 14, 2024Updated last year
- ☆388Feb 8, 2025Updated last year
- Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conv…☆631Mar 10, 2025Updated last year
- An open-source implementation for training LLaVA-NeXT.☆436Oct 23, 2024Updated last year
- Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.5, DeepSeek-R1, GLM-5, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, …☆12,956Updated this week
- 🔥🔥 LLaVA++: Extending LLaVA with Phi-3 and LLaMA-3 (LLaVA LLaMA-3, LLaVA Phi-3)☆848Aug 5, 2025Updated 7 months ago
- When do we not need larger vision models?☆413Feb 8, 2025Updated last year
- One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks☆3,830Updated this week
- Official repository for the paper PLLaVA☆676Jul 28, 2024Updated last year
- [CVPR 2025] LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant☆178Jul 7, 2025Updated 8 months ago
- [NeurIPS 2025 🔥] FakeVLM: Advancing Synthetic Image Detection through Explainable Multimodal Models and Fine-Grained Artifact Analysis☆122Sep 24, 2025Updated 5 months ago
- [CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型☆9,854Sep 22, 2025Updated 5 months ago
- Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks☆3,868Updated this week
- [ICCV 2025] LLaVA-CoT, a visual language model capable of spontaneous, systematic reasoning☆2,132Dec 12, 2025Updated 2 months ago
- A Next-Generation Training Engine Built for Ultra-Large MoE Models☆5,092Updated this week
- Visualizing the attention of vision-language models☆284Feb 28, 2025Updated last year
- ☆11Jan 8, 2025Updated last year
- A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.☆1,434Feb 11, 2026Updated 3 weeks ago
- [ICML 2025] Official PyTorch implementation of LongVU☆424May 8, 2025Updated 10 months ago
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆40Jan 4, 2024Updated 2 years ago
- Aligning LMMs with Factually Augmented RLHF☆393Nov 1, 2023Updated 2 years ago
- A fork to add multimodal model training to open-r1☆1,496Feb 8, 2025Updated last year
- An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.☆1,695Jan 10, 2026Updated 2 months ago
- [ICCV25 Highlight] The official implementation of the paper "LEGION: Learning to Ground and Explain for Synthetic Image Detection"☆73Oct 22, 2025Updated 4 months ago
- [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.☆24,500Aug 12, 2024Updated last year
- Cambrian-1 is a family of multimodal LLMs with a vision-centric design.☆1,988Nov 7, 2025Updated 4 months ago
- ☆190Feb 5, 2026Updated last month
- Latest Advances on Multimodal Large Language Models☆17,416Updated this week
- LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models☆166Sep 27, 2025Updated 5 months ago
- InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions☆2,922May 26, 2025Updated 9 months ago
- VILA is a family of state-of-the-art vision language models (VLMs) for diverse multimodal AI tasks across the edge, data center, and clou…☆3,771Nov 28, 2025Updated 3 months ago
- Harnessing 1.4M GPT4V-synthesized Data for A Lite Vision-Language Model☆281Jun 25, 2024Updated last year
- ☆17Apr 23, 2025Updated 10 months ago
- ☆13Feb 25, 2025Updated last year
- ☆13Jul 30, 2024Updated last year
- A RLHF Infrastructure for Vision-Language Models☆197Nov 15, 2024Updated last year
- Official repository for paper MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning(https://arxiv.org/abs/2406.17770).☆159Sep 27, 2024Updated last year