kyegomez / ScreenAI
Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"
☆323Updated last month
Alternatives and similar repositories for ScreenAI:
Users that are interested in ScreenAI are comparing it to the libraries listed below
- The model, data and code for the visual GUI Agent SeeClick☆330Updated 3 months ago
- [ICML'24] SeeAct is a system for generalist web agents that autonomously carry out tasks on any given website, with a focus on large mult…☆727Updated last month
- OS-ATLAS: A Foundation Action Model For Generalist GUI Agents☆293Updated 3 weeks ago
- ScreenQA dataset was introduced in the "ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots" paper. It contains ~86K …☆106Updated last month
- Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction☆254Updated this week
- 💻 A curated list of papers and resources for multi-modal Graphical User Interface (GUI) agents.☆547Updated this week
- ScreenAgent: A Computer Control Agent Driven by Visual Language Large Model (IJCAI-24)☆414Updated 3 months ago
- [ICLR'25 Oral] UGround: Universal GUI Visual Grounding for GUI Agents☆185Updated last week
- [NeurIPS'23 Spotlight] "Mind2Web: Towards a Generalist Agent for the Web"☆793Updated 7 months ago
- [ICLR 2025] A trinity of environments, tools, and benchmarks for general virtual agents☆195Updated 2 weeks ago
- 🦀️ CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents. https://crab.camel-ai.org/☆293Updated 3 months ago
- [CVPR 2025] Open-source, End-to-end, Vision-Language-Action model for GUI Agent & Computer Use.☆1,084Updated this week
- Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"☆912Updated last month
- The Screen Annotation dataset consists of pairs of mobile screenshots and their annotations. The annotations are in text format, and desc…☆62Updated last year
- This is a collection of resources for computer-use GUI agents, including videos, blogs, papers, and projects.☆264Updated this week
- GUI Grounding for Professional High-Resolution Computer Use☆134Updated 2 weeks ago
- VisualWebArena is a benchmark for multimodal agents.☆315Updated 4 months ago
- AI for all: Build the large graph of the language models☆263Updated 9 months ago
- An open-sourced end-to-end VLM-based GUI Agent☆815Updated 3 weeks ago
- GPT-4V in Wonderland: LMMs as Smartphone Agents☆133Updated 7 months ago
- WebLINX is a benchmark for building web navigation agents with conversational capabilities☆144Updated last month
- Code for "WebVoyager: WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models"☆661Updated last year
- Official implementation for "You Only Look at Screens: Multimodal Chain-of-Action Agents" (Findings of ACL 2024)☆221Updated 7 months ago
- Building Open LLM Web Agents with Self-Evolving Online Curriculum RL☆320Updated last month
- ☆111Updated last year
- ☆292Updated 11 months ago
- AI agent using GPT-4V(ision) capable of using a mouse/keyboard to interact with web UI☆1,029Updated 3 months ago
- LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA☆477Updated 2 months ago
- Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of multi-modal AI agents.☆617Updated this week
- GUICourse: From General Vision Langauge Models to Versatile GUI Agents☆103Updated 7 months ago