Alternatives and similar repositories for ScreenAI:

Users that are interested in ScreenAI are comparing it to the libraries listed below

• OSU-NLP-Group / SeeAct
  [ICML'24] SeeAct is a system for generalist web agents that autonomously carry out tasks on any given website, with a focus on large mult…
  ☆705Updated 2 weeks ago
• njucckevin / SeeClick
  The model, data and code for the visual GUI Agent SeeClick
  ☆312Updated 2 months ago
• niuzaisheng / ScreenAgent
  ScreenAgent: A Computer Control Agent Driven by Visual Language Large Model (IJCAI-24)
  ☆397Updated 2 months ago
• OSU-NLP-Group / Mind2Web
  [NeurIPS'23 Spotlight] "Mind2Web: Towards a Generalist Agent for the Web"
  ☆778Updated 6 months ago
• xlang-ai / aguvis
  Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction
  ☆221Updated last month
• OS-Copilot / OS-Atlas
  OS-ATLAS: A Foundation Action Model For Generalist GUI Agents
  ☆279Updated this week
• google-research-datasets / screen_qa
  ScreenQA dataset was introduced in the "ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots" paper. It contains ~86K …
  ☆103Updated last week
• showlab / ShowUI
  Open-source, End-to-end, Vision-Language-Action model for GUI Agent & Computer Use.
  ☆986Updated last week
• showlab / Awesome-GUI-Agent
  💻 A curated list of papers and resources for multi-modal Graphical User Interface (GUI) agents.
  ☆490Updated 3 weeks ago
• cooelf / Auto-GUI
  Official implementation for "You Only Look at Screens: Multimodal Chain-of-Action Agents" (Findings of ACL 2024)
  ☆219Updated 7 months ago
• microsoft / WindowsAgentArena
  Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of multi-modal AI agents.
  ☆584Updated 3 months ago
• zzxslp / MM-Navigator
  GPT-4V in Wonderland: LMMs as Smartphone Agents
  ☆132Updated 7 months ago
• web-arena-x / webarena
  Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"
  ☆864Updated last week
• OSU-NLP-Group / UGround
  [ICLR'25 Oral] UGround: Universal GUI Visual Grounding for GUI Agents
  ☆165Updated this week
• THUDM / AutoWebGLM
  An LLM-based Web Navigating Agent (KDD'24)
  ☆815Updated 4 months ago
• ltzheng / agent-studio
  [ICLR 2025] A trinity of environments, tools, and benchmarks for general virtual agents
  ☆192Updated 2 weeks ago
• ranpox / awesome-computer-use
  This is a collection of resources for computer-use GUI agents, including videos, blogs, papers, and projects.
  ☆221Updated this week
• McGill-NLP / weblinx
  WebLINX is a benchmark for building web navigation agents with conversational capabilities
  ☆141Updated last week
• NexaAI / octopus-v4
  AI for all: Build the large graph of the language models
  ☆254Updated 8 months ago
• js0nwu / webui
  ☆110Updated last year
• agent-husky / Husky-v1
  Code for Husky, an open-source language agent that solves complex, multi-step reasoning tasks. Husky v1 addresses numerical, tabular and …
  ☆336Updated 8 months ago
• google-research / android_world
  AndroidWorld is an environment and benchmark for autonomous agents
  ☆212Updated last week
• google-research-datasets / screen_annotation
  The Screen Annotation dataset consists of pairs of mobile screenshots and their annotations. The annotations are in text format, and desc…
  ☆60Updated 11 months ago
• MinorJerry / WebVoyager
  Code for "WebVoyager: WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models"
  ☆615Updated 11 months ago
• web-arena-x / visualwebarena
  VisualWebArena is a benchmark for multimodal agents.
  ☆295Updated 3 months ago
• microsoft / SoM
  Set-of-Mark Prompting for GPT-4V and LMMs
  ☆1,280Updated 6 months ago
• simular-ai / Agent-S
  Agent S: an open agentic framework that uses computers like a human
  ☆803Updated 3 weeks ago
• LLaVA-VL / LLaVA-Plus-Codebase
  LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills
  ☆722Updated last year
• ddupont808 / GPT-4V-Act
  AI agent using GPT-4V(ision) capable of using a mouse/keyboard to interact with web UI
  ☆1,020Updated 2 months ago
• AriaUI / Aria-UI
  Open-sourced, Fast and Context-aware Action Grounding from GUI Instructions for GUI/Computer-use Agents
  ☆320Updated last week