Alternatives and similar repositories for widget-caption

Users that are interested in widget-caption are comparing it to the libraries listed below

• google-research-datasets / screen2words
  The dataset includes screen summaries that describes Android app screenshot's functionalities. It is used for training and evaluation of …
  ☆60Updated 4 years ago
• js0nwu / webui
  ☆123Updated last year
• aburns4 / MoTIF
  Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments
  ☆60Updated last year
• google-research-datasets / screen_qa
  ScreenQA dataset was introduced in the "ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots" paper. It contains ~86K …
  ☆129Updated 8 months ago
• google-research-datasets / rico_semantics
  Consists of ~500k human annotations on the RICO dataset identifying various icons based on their shapes and semantics, and associations b…
  ☆31Updated last year
• google-research-datasets / seq2act
  This repository contains the opensource version of the datasets were used for different parts of training and testing of models that grou…
  ☆32Updated 5 years ago
• google-research-datasets / screen_annotation
  The Screen Annotation dataset consists of pairs of mobile screenshots and their annotations. The annotations are in text format, and desc…
  ☆78Updated last year
• cooelf / Auto-GUI
  Official implementation for "You Only Look at Screens: Multimodal Chain-of-Action Agents" (Findings of ACL 2024)
  ☆253Updated last year
• google-research-datasets / clay
  The dataset includes UI object type labels (e.g., BUTTON, IMAGE, CHECKBOX) that describes the semantic type of an UI object on Android ap…
  ☆53Updated 3 years ago
• google-research-datasets / uibert
  It includes two datasets that are used in the downstream tasks for evaluating UIBert: App Similar Element Retrieval data and Visual Item …
  ☆46Updated 4 years ago
• MulongXie / GUI-Perceptual-Grouping
  Recognize graphic user interface layout through grouping GUI elements according to their visual attributes
  ☆47Updated 3 years ago
• chenjshnn / Object-Detection-for-Graphical-User-Interface
  Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?
  ☆127Updated last year
• njucckevin / SeeClick
  The model, data and code for the visual GUI Agent SeeClick
  ☆432Updated 3 months ago
• datadrivendesign / semantic-icon-classifier
  ☆36Updated 2 years ago
• RUCBM / GUICourse
  GUICourse: From General Vision Langauge Models to Versatile GUI Agents
  ☆132Updated last year
• luileito / enrico
  A curated mobile app design database
  ☆63Updated 4 years ago
• IMNearth / CoAT
  Official implementation for "Android in the Zoo: Chain-of-Action-Thought for GUI Agents" (Findings of EMNLP 2024)
  ☆94Updated last year
• web-arena-x / visualwebarena
  VisualWebArena is a benchmark for multimodal agents.
  ☆395Updated 11 months ago
• zzxslp / MM-Navigator
  GPT-4V in Wonderland: LMMs as Smartphone Agents
  ☆135Updated last year
• NiteshMethani / PlotQA
  Dataset introduced in PlotQA: Reasoning over Scientific Plots
  ☆80Updated 2 years ago
• alipay / mobile-agent
  ☆44Updated last year
• vis-nlp / ChartQA
  ☆224Updated 6 months ago
• nttmdlab-nlp / SlideVQA
  SlideVQA: A Dataset for Document Visual Question Answering on Multiple Images (AAAI2023)
  ☆98Updated 7 months ago
• sbunian / VINS
  VINS: Visual Search for Mobile User Interface Design
  ☆46Updated 4 years ago
• bhargaviparanjape / language-programmes
  ☆173Updated 2 years ago
• rubenpt91 / MP-DocVQA-Framework
  ☆67Updated last year
• magicgh / Self-MAP
  [ACL 2024] On the Multi-turn Instruction Following for Conversational Web Agents
  ☆16Updated last year
• X-LANCE / META-GUI-baseline
  [EMNLP 2022] The baseline code for META-GUI dataset
  ☆14Updated last year
• vis-nlp / Chart-to-text
  ☆117Updated last year
• TIGER-AI-Lab / Program-of-Thoughts
  Data and Code for Program of Thoughts [TMLR 2023]
  ☆292Updated last year