VisionTasker introduces a novel two-stage framework combining vision-based UI understanding and LLM task planning for mobile task automation in a step-by-step manner.
☆113Jul 18, 2025Updated last year
Alternatives and similar repositories for VisionTasker
Users that are interested in VisionTasker are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Recognize graphic user interface layout through grouping GUI elements according to their visual attributes☆52Jun 17, 2022Updated 4 years ago
- A mobile GUI search engine using a vision-language model☆15May 5, 2025Updated last year
- LlamaTouch: A Faithful and Scalable Testbed for Mobile UI Task Automation☆70Aug 9, 2024Updated 2 years ago
- ☆25Oct 11, 2024Updated 2 years ago
- ☆12Aug 8, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- [ICCV 2025] GUIOdyssey is a comprehensive dataset for training and evaluating cross-app navigation agents. GUIOdyssey consists of 8,834 e…☆162Jan 3, 2026Updated 9 months ago
- Source code for the paper "Empowering LLM to use Smartphone for Intelligent Task Automation"☆496Mar 22, 2024Updated 2 years ago
- This is the official repository of the paper "Atomic-to-Compositional Generalization for Mobile Agents with A New Benchmark and Schedulin…☆15Jul 27, 2025Updated last year
- Official implementation for "Android in the Zoo: Chain-of-Action-Thought for GUI Agents" (Findings of EMNLP 2024)☆104Oct 14, 2024Updated 2 years ago
- UI auto test framework based on YOLO to recognize elements, less code, less maintenance, cross platform, cross project / 基于YOLO的UI层自动化测试框…☆15Sep 16, 2026Updated 3 weeks ago
- Code for paper: "Executing Arithmetic: Fine-Tuning Large Language Models as Turing Machines"☆10Oct 11, 2024Updated 2 years ago
- The model, data and code for the visual GUI Agent SeeClick☆495Jul 13, 2025Updated last year
- ☆33Sep 27, 2024Updated 2 years ago
- Consists of ~500k human annotations on the RICO dataset identifying various icons based on their shapes and semantics, and associations b…☆36Jun 27, 2024Updated 2 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- The dataset includes widget captions that describes UI element's functionalities. It is used for training and evaluation of the widget ca…☆23Jun 24, 2021Updated 5 years ago
- Code repo for "Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding"☆32May 12, 2026Updated 5 months ago
- UICrit is a dataset containing human-generated natural language design critiques, corresponding bounding boxes for each critique, and des…☆28Nov 19, 2024Updated last year
- This repo is for our submission for ICSE 2025.☆20Jun 12, 2024Updated 2 years ago
- ☆35Jun 20, 2024Updated 2 years ago
- ☆26Jun 1, 2026Updated 4 months ago
- Exercises for the Dafny Tutorial☆14May 21, 2018Updated 8 years ago
- 🎮Manipulates mobile phones just like how you would. Official code for "MobA: Multifaceted Memory-Enhanced Adaptive Planning for Efficien…☆28Oct 10, 2025Updated last year
- [ICML 2026] GUIEvalKit: Open-source Evaluation Toolkit for GUI Agents☆25Feb 26, 2026Updated 7 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- 💻 A curated list of papers and resources for multi-modal Graphical User Interface (GUI) agents.☆1,218Aug 17, 2025Updated last year
- [NeurIPS 2025 Spotlight] Official repository for "Web-Shepherd: Advancing PRMs for Reinforcing Web Agents"☆61May 21, 2025Updated last year
- (ICLR 2025) The Official Code Repository for GUI-World.☆70Aug 2, 2026Updated 2 months ago
- [ACL 2025] GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agent☆68May 28, 2025Updated last year
- Evaluating Safety of Autonomous Agents in Mobile Device Control (AAAI 2026 AI Alignment Track)☆37Jan 28, 2026Updated 8 months ago
- ☆71Mar 30, 2025Updated last year
- Unblind Your Apps: Predicting Natural-Language Labels for Mobile GUI Components by Deep Learning☆48Nov 16, 2023Updated 2 years ago
- This repository holds the data and code for the AndroR2 dataset of manually-reproduced bug reports for Android apps☆27Jun 11, 2021Updated 5 years ago
- 非机动车头盔佩戴检测☆12Jan 20, 2026Updated 8 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆36Jun 5, 2025Updated last year
- [ACL 2025] Code and data for OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis☆192Oct 8, 2025Updated last year
- Official implementation of AppAgentX: Evolving GUI Agents as Proficient Smartphone Users☆680Apr 15, 2025Updated last year
- ☆48Apr 11, 2024Updated 2 years ago
- Under construction☆14Jan 15, 2025Updated last year
- Awesome GUI Agent Paper List☆910Oct 7, 2026Updated last week
- Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?☆128Feb 20, 2024Updated 2 years ago