Alternatives and similar repositories for SoM:

Users that are interested in SoM are comparing it to the libraries listed below

• LLaVA-VL / LLaVA-Plus-Codebase
  LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills
  ☆720Updated 11 months ago
• ddupont808 / GPT-4V-Act
  AI agent using GPT-4V(ision) capable of using a mouse/keyboard to interact with web UI
  ☆1,008Updated last month
• roboflow / maestro
  streamline the fine-tuning process for multimodal models: PaliGemma, Florence-2, and Qwen2-VL
  ☆1,427Updated this week
• OSU-NLP-Group / SeeAct
  [ICML'24] SeeAct is a system for generalist web agents that autonomously carry out tasks on any given website, with a focus on large mult…
  ☆686Updated this week
• microsoft / MM-REACT
  Official repo for MM-REACT
  ☆940Updated 11 months ago
• OpenGVLab / VisionLLM
  VisionLLM Series
  ☆977Updated 2 weeks ago
• OSU-NLP-Group / Mind2Web
  [NeurIPS'23 Spotlight] "Mind2Web: Towards a Generalist Agent for the Web"
  ☆759Updated 5 months ago
• OptimalScale / DetGPT
  ☆762Updated 5 months ago
• shikras / shikra
  ☆756Updated 6 months ago
• AILab-CVC / GPT4Tools
  GPT4Tools is an intelligent system that can automatically decide, control, and utilize different visual foundation models, allowing the u…
  ☆763Updated last year
• penghao-wu / vstar
  PyTorch Implementation of "V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs"
  ☆556Updated last year
• LLaVA-VL / LLaVA-Interactive-Demo
  LLaVA-Interactive-Demo
  ☆360Updated 5 months ago
• dvlab-research / LISA
  Project Page for "LISA: Reasoning Segmentation via Large Language Model"
  ☆1,949Updated 2 weeks ago
• jshilong / GPT4RoI
  GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest
  ☆521Updated 7 months ago
• Alpha-VLLM / LLaMA2-Accessory
  An Open-source Toolkit for LLM Development
  ☆2,747Updated this week
• THUDM / AgentBench
  A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
  ☆2,326Updated 2 months ago
• magic-research / bubogpt
  BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs
  ☆506Updated last year
• dvlab-research / LLaMA-VID
  LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models (ECCV 2024)
  ☆758Updated 5 months ago
• SkunkworksAI / BakLLaVA
  ☆704Updated 10 months ago
• dongyh20 / Octopus
  🐙Octopus, an embodied vision-language model trained with RLEF, emerging superior in embodied visual planning and programming.
  ☆278Updated 7 months ago
• web-arena-x / webarena
  Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"
  ☆812Updated 3 weeks ago
• Meituan-AutoML / MobileVLM
  Strong and Open Vision Language Assistant for Mobile Devices
  ☆1,106Updated 9 months ago
• showlab / VLog
  Transform Video as a Document with ChatGPT, CLIP, BLIP2, GRIT, Whisper, LangChain.
  ☆549Updated last year
• PKU-YuanGroup / Video-LLaVA
  【EMNLP 2024🔥】Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
  ☆3,113Updated last month
• facebookresearch / MetaCLIP
  ICLR2024 Spotlight: curation/training code, metadata, distribution and pre-trained models for MetaCLIP; CVPR 2024: MoDE: CLIP Data Expert…
  ☆1,337Updated last month
• THUDM / AgentTuning
  AgentTuning: Enabling Generalized Agent Abilities for LLMs
  ☆1,383Updated last year
• Luodian / RelateAnything
  Relate Anything Model is capable of taking an image as input and utilizing SAM to identify the corresponding mask within the image.
  ☆448Updated last year
• showlab / Image2Paragraph
  [A toolbox for fun.] Transform Image into Unique Paragraph with ChatGPT, BLIP2, OFA, GRIT, Segment Anything, ControlNet.
  ☆800Updated last year
• open-mmlab / Multimodal-GPT
  Multimodal-GPT
  ☆1,488Updated last year
• showlab / ShowUI
  Open-source, End-to-end, Vision-Language-Action model for GUI Agent & Computer Use.
  ☆841Updated this week