The dataset includes screen summaries that describes Android app screenshot's functionalities. It is used for training and evaluation of the screen2words models (our paper accepted by UIST'21 will be linked soon).
☆67Jul 27, 2021Updated 4 years ago
Alternatives and similar repositories for screen2words
Users that are interested in screen2words are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- The dataset includes widget captions that describes UI element's functionalities. It is used for training and evaluation of the widget ca…☆23Jun 24, 2021Updated 5 years ago
- Screen2Vec is a new self-supervised technique for generating more comprehensive semantic embeddings of GUI screens and components using t…☆85Feb 3, 2025Updated last year
- It includes two datasets that are used in the downstream tasks for evaluating UIBert: App Similar Element Retrieval data and Visual Item …☆48Aug 2, 2021Updated 4 years ago
- The Screen Annotation dataset consists of pairs of mobile screenshots and their annotations. The annotations are in text format, and desc…☆93Mar 7, 2024Updated 2 years ago
- A curated mobile app design database☆72Sep 27, 2021Updated 4 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- The dataset includes UI object type labels (e.g., BUTTON, IMAGE, CHECKBOX) that describes the semantic type of an UI object on Android ap…☆54Jan 14, 2022Updated 4 years ago
- ☆33Sep 27, 2024Updated last year
- ☆132Dec 4, 2023Updated 2 years ago
- Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments☆61Aug 19, 2024Updated last year
- A mobile GUI search engine using a vision-language model☆15May 5, 2025Updated last year
- ScreenQA dataset was introduced in the "ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots" paper. It contains ~86K …☆151Feb 7, 2025Updated last year
- ☆24Oct 11, 2024Updated last year
- Code repo for "Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding"☆31May 12, 2026Updated 2 months ago
- The official repository of "SmartAgent: Chain-of-User-Thought for Embodied Personalized Agent in Cyber World".☆27Aug 20, 2025Updated 11 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Owl Eyes: Spotting UI Display Issues via Visual Understanding☆22Nov 15, 2021Updated 4 years ago
- Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"☆386Updated this week
- Crawl github data using API and no-API☆12Jul 4, 2017Updated 9 years ago
- We release the UICaption dataset. The dataset consists of UI images (icons and screenshots) and associated text descriptions. This datase…☆42Nov 29, 2022Updated 3 years ago
- This repository holds the data and code for the AndroR2 dataset of manually-reproduced bug reports for Android apps☆26Jun 11, 2021Updated 5 years ago
- ☆36Jan 22, 2019Updated 7 years ago
- [ICML 2026] GUIEvalKit: Open-source Evaluation Toolkit for GUI Agents☆23Feb 26, 2026Updated 4 months ago
- The official GitHub page for ''What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Ins…☆19Nov 10, 2023Updated 2 years ago
- Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?☆128Feb 20, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆36May 29, 2025Updated last year
- Overview of Clone Detection Tools for Java☆14Aug 23, 2025Updated 11 months ago
- Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"☆68Oct 19, 2024Updated last year
- LINEBot☆13Apr 7, 2025Updated last year
- Consists of ~500k human annotations on the RICO dataset identifying various icons based on their shapes and semantics, and associations b…☆36Jun 27, 2024Updated 2 years ago
- (ICLR 2025) The Official Code Repository for GUI-World.☆69Dec 18, 2024Updated last year
- ☆15Sep 8, 2025Updated 10 months ago
- ☆102Dec 22, 2023Updated 2 years ago
- A customized version of Ella used in the paper `An Empirical Study of Android Test Generation Tools in Industrial Cases`.☆10Aug 19, 2020Updated 5 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Game UI Glitch Detection via Bug Understanding☆12Jul 31, 2021Updated 4 years ago
- Repository of paper: Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Models☆37Sep 19, 2023Updated 2 years ago
- [ECCV 2024 Oral] GazeXplain - Official PyTorch Implementation☆17Feb 24, 2025Updated last year
- ☆10Aug 28, 2020Updated 5 years ago
- This is the official repository of the paper "Atomic-to-Compositional Generalization for Mobile Agents with A New Benchmark and Schedulin…☆14Jul 27, 2025Updated 11 months ago
- The model, data and code for the visual GUI Agent SeeClick☆493Jul 13, 2025Updated last year
- Generation and evaluation of a decision tree using reinforcement learning.☆11Dec 8, 2022Updated 3 years ago