This repository contains the opensource version of the datasets were used for different parts of training and testing of models that ground natural language to UI actions as described in the paper: "Mapping Natural Language Instructions to Mobile UI Action Sequences" by Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, and Jason Baldridge, which is acc…
☆34Aug 20, 2020Updated 5 years ago
Alternatives and similar repositories for seq2act
Users that are interested in seq2act are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Seq2act: Mapping Natural Language Instructions to Mobile UI Action Sequences from Google research☆15Jul 13, 2020Updated 5 years ago
- Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments☆61Aug 19, 2024Updated last year
- The dataset includes UI object type labels (e.g., BUTTON, IMAGE, CHECKBOX) that describes the semantic type of an UI object on Android ap…☆54Jan 14, 2022Updated 4 years ago
- A Universal Platform for Training and Evaluation of Mobile Interaction☆62Sep 24, 2025Updated 7 months ago
- ☆47Apr 11, 2024Updated 2 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Under construction☆13Jan 15, 2025Updated last year
- It includes two datasets that are used in the downstream tasks for evaluating UIBert: App Similar Element Retrieval data and Visual Item …☆48Aug 2, 2021Updated 4 years ago
- LlamaTouch: A Faithful and Scalable Testbed for Mobile UI Task Automation☆69Aug 9, 2024Updated last year
- A dataset consisting of 502 English dialogs with 12,000 annotated utterances between a user and an assistant discussing movie preferences…☆28Jan 20, 2021Updated 5 years ago
- An environment for mobile angets to interact with realistic android device or android emulator☆13Jul 19, 2024Updated last year
- Benchmarking Mobile Device Control Agents across Diverse Configurations (ICLR 2024 workshop GenAI4DM spotlight presentation; CoLLAs 2025)☆35Jul 21, 2025Updated 10 months ago
- A curated mobile app design database☆70Sep 27, 2021Updated 4 years ago
- Recognize graphic user interface layout through grouping GUI elements according to their visual attributes☆50Jun 17, 2022Updated 3 years ago
- The dataset includes widget captions that describes UI element's functionalities. It is used for training and evaluation of the widget ca…☆23Jun 24, 2021Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation☆65Jul 11, 2025Updated 10 months ago
- UICrit is a dataset containing human-generated natural language design critiques, corresponding bounding boxes for each critique, and des…☆25Nov 19, 2024Updated last year
- ☆32Sep 27, 2024Updated last year
- MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding☆78Feb 27, 2025Updated last year
- [ACL'24] WebCiteS: Attributed Query-Focused Summarization on Chinese Web Search Results with Citations☆13Sep 11, 2024Updated last year
- Game UI Glitch Detection via Bug Understanding☆12Jul 31, 2021Updated 4 years ago
- RL research on Android devices.☆1,217May 13, 2026Updated last week
- Most of the React Native styling material in one page☆14Aug 19, 2016Updated 9 years ago
- ☆23Aug 29, 2023Updated 2 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Owl Eyes: Spotting UI Display Issues via Visual Understanding☆12Jul 31, 2020Updated 5 years ago
- ☆12Aug 24, 2023Updated 2 years ago
- ☆35Mar 24, 2023Updated 3 years ago
- [SCIS] MULTI-Benchmark: Multimodal Understanding Leaderboard with Text and Images☆45Nov 19, 2025Updated 6 months ago
- Conv Net for identifying GUI componenets from screenshots using Tensorflow☆12Mar 24, 2023Updated 3 years ago
- ☆10Nov 9, 2023Updated 2 years ago
- This is the official repository of the paper "Atomic-to-Compositional Generalization for Mobile Agents with A New Benchmark and Schedulin…☆14Jul 27, 2025Updated 9 months ago
- 《A fast and elitist multi-objective genetic algorithm: NSGA-II》☆23Mar 6, 2023Updated 3 years ago
- GPT* - Training faster small transformers using ALiBi, Parallel Residual Connections and more!☆20Oct 29, 2022Updated 3 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆10Feb 8, 2021Updated 5 years ago
- Бенчмарк для оценки способности языковых моделей решать математические и физические задачи на русском языке☆22Nov 14, 2025Updated 6 months ago
- Explore Android apps like human.☆133Feb 18, 2023Updated 3 years ago
- ☆17Oct 30, 2023Updated 2 years ago
- A mobile GUI search engine using a vision-language model☆14May 5, 2025Updated last year
- ☆11Mar 24, 2023Updated 3 years ago
- ☆53Jan 24, 2024Updated 2 years ago