The dataset includes screen summaries that describes Android app screenshot's functionalities. It is used for training and evaluation of the screen2words models (our paper accepted by UIST'21 will be linked soon).
☆67Jul 27, 2021Updated 5 years ago
Alternatives and similar repositories for screen2words
Users that are interested in screen2words are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- The dataset includes widget captions that describes UI element's functionalities. It is used for training and evaluation of the widget ca…☆23Jun 24, 2021Updated 5 years ago
- Screen2Vec is a new self-supervised technique for generating more comprehensive semantic embeddings of GUI screens and components using t…☆86Feb 3, 2025Updated last year
- The Screen Annotation dataset consists of pairs of mobile screenshots and their annotations. The annotations are in text format, and desc…☆94Mar 7, 2024Updated 2 years ago
- VINS: Visual Search for Mobile User Interface Design☆54Jan 9, 2021Updated 5 years ago
- It includes two datasets that are used in the downstream tasks for evaluating UIBert: App Similar Element Retrieval data and Visual Item …☆48Aug 2, 2021Updated 5 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A curated mobile app design database☆77Sep 27, 2021Updated 5 years ago
- Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments☆61Aug 19, 2024Updated 2 years ago
- This repository contains the opensource version of the datasets were used for different parts of training and testing of models that grou…☆36Aug 20, 2020Updated 6 years ago
- ScreenQA dataset was introduced in the "ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots" paper. It contains ~86K …☆153Feb 7, 2025Updated last year
- ☆17May 14, 2024Updated 2 years ago
- The dataset includes UI object type labels (e.g., BUTTON, IMAGE, CHECKBOX) that describes the semantic type of an UI object on Android ap…☆54Jan 14, 2022Updated 4 years ago
- [WWW2024 Oral] Harnessing Multi-Role Capabilities of Large Language Models for Open-Domain Question Answering☆15Apr 22, 2025Updated last year
- ☆33Sep 27, 2024Updated 2 years ago
- An accurate GUI element detection approach based on old-fashioned CV algorithms [Upgraded on 5/July/2021]☆555Nov 8, 2023Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆132Dec 4, 2023Updated 2 years ago
- A mobile GUI search engine using a vision-language model☆15May 5, 2025Updated last year
- ☆17Oct 30, 2023Updated 3 years ago
- Seq2act: Mapping Natural Language Instructions to Mobile UI Action Sequences from Google research☆15Jul 13, 2020Updated 6 years ago
- Owl Eyes: Spotting UI Display Issues via Visual Understanding☆22Nov 15, 2021Updated 4 years ago
- Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"☆387Oct 26, 2026Updated last week
- The official GitHub page for ''What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Ins…☆18Nov 10, 2023Updated 2 years ago
- [ICCV 2025] GUIOdyssey is a comprehensive dataset for training and evaluating cross-app navigation agents. GUIOdyssey consists of 8,834 e…☆162Jan 3, 2026Updated 9 months ago
- We release the UICaption dataset. The dataset consists of UI images (icons and screenshots) and associated text descriptions. This datase…☆42Nov 29, 2022Updated 3 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Overview of Clone Detection Tools for Java☆14Aug 23, 2025Updated last year
- Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?☆128Feb 20, 2024Updated 2 years ago
- This repository holds the data and code for the AndroR2 dataset of manually-reproduced bug reports for Android apps☆28Jun 11, 2021Updated 5 years ago
- ☆14May 29, 2025Updated last year
- ☆12Dec 12, 2021Updated 4 years ago
- Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"☆68Oct 19, 2024Updated 2 years ago
- Differences and similarities between TVCG and VIS papers. Also, crowdsourced submission data for TVCG. Enter anonymous submission data at…☆26Feb 7, 2022Updated 4 years ago
- Code for paper "Prompt Engineering a Prompt Engineer" (https://arxiv.org/abs/2311.05661)☆12Aug 1, 2024Updated 2 years ago
- SWIRL, a staged workflow for interleaved reinforcement learning designed for multi-agent systems.☆14Oct 26, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- (ICLR 2025) The Official Code Repository for GUI-World.☆70Aug 2, 2026Updated 3 months ago
- Consists of ~500k human annotations on the RICO dataset identifying various icons based on their shapes and semantics, and associations b…☆36Jun 27, 2024Updated 2 years ago
- 📖 UI/UX context detection engine☆12Jan 3, 2021Updated 5 years ago
- ☆102Dec 22, 2023Updated 2 years ago
- Automatically instrument gradle-based applications and generate code coverage reports☆13Apr 1, 2021Updated 5 years ago
- This Repository is Research mode view with Unity in HoloLens☆14Apr 14, 2018Updated 8 years ago
- ☆111Sep 11, 2025Updated last year