WatchTower-Liu / VLM-learningLinks
Building a VLM model starts from the basic module.
☆16Updated last year
Alternatives and similar repositories for VLM-learning
Users that are interested in VLM-learning are comparing it to the libraries listed below
Sorting:
- ☆26Updated 10 months ago
- Build a simple basic multimodal large model from scratch. 从零搭建一个简单的基础多模态大模型🤖☆41Updated last year
- 从零到一实现了一个多模态大模型，并命名为Reyes（睿视），R：睿，eyes：眼。Reyes的参数量为8B，视觉编码器使用的是InternViT-300M-448px-V2_5,语言模型侧使用的是Qwen2.5-7B-Instruct，Reyes也通过一个两层MLP投影层连…☆14Updated 4 months ago
- A multimodal image search engine built on the GME model, capable of handling diverse input types. Whether you're querying with text, imag…☆40Updated 2 weeks ago
- 训练一个对中文支持更好的LLaVA模型，并开源训练代码和数据。☆61Updated 9 months ago
- Research Code for Multimodal-Cognition Team in Ant Group☆153Updated last month
- ☆57Updated last year
- This project aims to collect and collate various datasets for multimodal large model training, including but not limited to pre-training …☆48Updated last month
- 多模态 MM +Chat 合集☆268Updated last month
- The official repo for “TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding”.☆40Updated 9 months ago
- The huggingface implementation of Fine-grained Late-interaction Multi-modal Retriever.☆91Updated 3 weeks ago
- WWW2025 Multimodal Intent Recognition for Dialogue Systems Challenge☆120Updated 7 months ago
- DeepSpeed Tutorial☆97Updated 10 months ago
- Train InternViT-6B in MMSegmentation and MMDetection with DeepSpeed☆93Updated 8 months ago
- Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines☆126Updated 7 months ago
- A Token-level Text Image Foundation Model for Document Understanding☆96Updated last month
- The official code of "Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs"☆75Updated last month
- Exploring Efficient Fine-Grained Perception of Multimodal Large Language Models☆62Updated 7 months ago
- The official code for NeurIPS 2024 paper: Harmonizing Visual Text Comprehension and Generation☆129Updated 7 months ago
- [CVPR 2024] LION: Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge☆145Updated 11 months ago
- ☆42Updated 4 months ago
- 【ArXiv】PDF-Wukong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Sampling☆120Updated 3 weeks ago
- A Survey of Multimodal Retrieval-Augmented Generation☆18Updated 2 months ago
- The official implementation of RAR☆88Updated last year
- Workshop on Foundation Model 1st foundation model challenge Track1 codebase (Open TransMind v1.0)☆18Updated 2 years ago
- MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval☆189Updated last month
- Multimodal Open-O1 (MO1) is designed to enhance the accuracy of inference models by utilizing a novel prompt-based approach. This tool wo…☆29Updated 9 months ago
- Official PyTorch Implementation of MLLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced …☆78Updated 7 months ago
- 八戒-Chat是利用《西游记》剧本中所有关于猪八戒的台词和语句，以及Chat-GPT-3.5生成的相关问题结果，基于Internlm进行QLoRA微调得到的模仿猪八戒语气的聊天语言模型。☆23Updated 11 months ago
- ☆85Updated last year