Alternatives and similar repositories for SoM

Users that are interested in SoM are comparing it to the libraries listed below

• LLaVA-VL / LLaVA-Plus-Codebase
  LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills
  ☆749Updated last year
• microsoft / MM-REACT
  Official repo for MM-REACT
  ☆953Updated last year
• ddupont808 / GPT-4V-Act
  AI agent using GPT-4V(ision) capable of using a mouse/keyboard to interact with web UI
  ☆1,045Updated 7 months ago
• OSU-NLP-Group / SeeAct
  [ICML'24] SeeAct is a system for generalist web agents that autonomously carry out tasks on any given website, with a focus on large mult…
  ☆760Updated 5 months ago
• OSU-NLP-Group / Mind2Web
  [NeurIPS'23 Spotlight] "Mind2Web: Towards a Generalist Agent for the Web" -- the first LLM-based web agent and benchmark for generalist w…
  ☆841Updated 3 months ago
• penghao-wu / vstar
  PyTorch Implementation of "V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs"
  ☆644Updated last year
• OpenGVLab / VisionLLM
  VisionLLM Series
  ☆1,084Updated 4 months ago
• OptimalScale / DetGPT
  ☆779Updated 11 months ago
• LLaVA-VL / LLaVA-Interactive-Demo
  LLaVA-Interactive-Demo
  ☆374Updated 11 months ago
• dvlab-research / LLaMA-VID
  LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models (ECCV 2024)
  ☆820Updated 11 months ago
• AILab-CVC / GPT4Tools
  GPT4Tools is an intelligent system that can automatically decide, control, and utilize different visual foundation models, allowing the u…
  ☆775Updated last year
• mbzuai-oryx / groundingLMM
  [CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses tha…
  ☆893Updated last month
• kyegomez / ScreenAI
  Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"
  ☆351Updated 3 months ago
• mbzuai-oryx / Video-ChatGPT
  [ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the cap…
  ☆1,396Updated 3 months ago
• open-mmlab / Multimodal-GPT
  Multimodal-GPT
  ☆1,506Updated 2 years ago
• Meituan-AutoML / MobileVLM
  Strong and Open Vision Language Assistant for Mobile Devices
  ☆1,239Updated last year
• dvlab-research / LISA
  Project Page for "LISA: Reasoning Segmentation via Large Language Model"
  ☆2,296Updated 5 months ago
• web-arena-x / webarena
  Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"
  ☆1,048Updated 5 months ago
• DirtyHarryLYL / LLM-in-Vision
  Recent LLM-based CV and related works. Welcome to comment/contribute!
  ☆868Updated 4 months ago
• shikras / shikra
  ☆783Updated last year
• PKU-YuanGroup / Video-LLaVA
  【EMNLP 2024🔥】Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
  ☆3,306Updated 7 months ago
• SkunkworksAI / BakLLaVA
  ☆710Updated last year
• BAAI-DCAI / Bunny
  A family of lightweight multimodal models.
  ☆1,024Updated 7 months ago
• DAMO-NLP-SG / Video-LLaMA
  [EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
  ☆3,031Updated last year
• showlab / ShowUI
  [CVPR 2025] Open-source, End-to-end, Vision-Language-Action model for GUI Agent & Computer Use.
  ☆1,370Updated last month
• mbzuai-oryx / LLaVA-pp
  🔥🔥 LLaVA++: Extending LLaVA with Phi-3 and LLaMA-3 (LLaVA LLaMA-3, LLaVA Phi-3)
  ☆840Updated last year
• Alpha-VLLM / LLaMA2-Accessory
  An Open-source Toolkit for LLM Development
  ☆2,787Updated 6 months ago
• IDEA-Research / Grounding-DINO-1.5-API
  Grounding DINO 1.5: IDEA Research's Most Capable Open-World Object Detection Model Series
  ☆983Updated 5 months ago
• yunlong10 / Awesome-LLMs-for-Video-Understanding
  🔥🔥🔥Latest Papers, Codes and Datasets on Vid-LLMs.
  ☆2,491Updated this week
• jshilong / GPT4RoI
  GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest
  ☆535Updated last month