Subgoal-aware Memory Graph for GUI Agent
GUI agents struggle in long-horizon mobile tasks because they must repeatedly re-discover intermediate workflows from scratch. This paper proposes a subgoal-aware memory graph that segments past trajectories into contiguous action sequences sharing a single intent, then merges visually similar boundary screenshots into shared graph nodes and attaches annotated subgoal transitions as directed edges. The core claim is that this intermediate granularity is more reusable than either raw actions or full trajectories, which the authors test on a challenging mobile benchmark.
The paper identifies a real representational gap between action-level and trajectory-level memory, proposes a clean graph construction pipeline, and shows that subgoal granularity outperforms both extremes in an internal ablation. However, the empirical support is thin: only 13 tasks from a single benchmark split are evaluated, with one backbone model and no error bars. The absence of direct numerical comparisons to prior memory methods such as HyMEM or GUI-Explorer also weakens the argument that subgoal graphs are the best practical choice.
The subgoal abstraction is well motivated, and the ablation design cleanly isolates the contribution of granularity and structure. The graph topology itself provides a clear benefit: replacing it with a flat store drops success rate to 30.77%, and neither action-level (38.46%) nor task-level (36.75%) memory reaches the subgoal graph. The pipeline—embedding-based state merging with thresholding, edge deduplication, and top-$k$ retrieval—is described concretely and is easy to follow.
The evaluation is far too narrow to support broad claims. The authors test on only 13 English single-app level-3 tasks from SPA-Bench, a tiny slice of mobile interaction space. Results are averaged over three runs yet no standard deviations are reported, so the stability of the 45.30% versus 34.19% gap is unknown. Only one backbone, Qwen3-VL-8B-Instruct, is used, leaving generalization across model families unexplored. Crucially, the paper omits essential implementation details: which LLM performs subgoal segmentation and metadata extraction, what prompts are used, and how large the trajectory corpus is.
A specific interpretive issue also weakens the evidence. The authors state that 'neither action-level nor task-level memory yields meaningful gains over the baseline,' but action-level graph memory achieves 38.46% versus the baseline 34.19%—a smaller gain than subgoal's, but not obviously meaningless. Without significance testing, this framing is unsupported.
The internal ablations are fair for demonstrating that subgoal granularity and graph structure help within the authors' specific setup, but the paper offers no head-to-head comparison against prior work. HyMEM and GUI-Explorer are discussed only qualitatively, with no shared benchmark numbers, so readers cannot assess whether the proposed graph is actually superior to these alternatives. The absolute success rate remains below 50%, indicating substantial room for improvement. Because the evaluated task set is so small and no statistical testing is performed, the evidence supports the specific trend in the ablation but not the broader claim that subgoal graphs are the right general solution for GUI memory.
Reproducibility is limited. No code or prompt templates are released. While some hyperparameters are disclosed—$\tau = 0.9$, $\theta = 0.9$, $k = 1$—critical choices are missing: the identity of the LLM used for segmentation and annotation, the prompts for subgoal metadata extraction, the size and source of the trajectory corpus, and the exact formatting of retrieved memory blocks in the agent context. The paper notes that construction takes 4 GPU-hours and retrieval adds ~0.5 s, but without knowing the corpus scale these numbers are uninformative. These gaps would block an independent reproduction.
GUI agents aim to automate complex user tasks by interacting with mobile interfaces, but ef- fective decision-making in long-horizon set- tings requires reusing prior experience rather than relying solely on the environment ob- servations. Existing memory-based methods typically store experience either at the single- action level or as entire trajectories. Single- step actions are often too fine-grained to carry meaningful semantics, since the success of an individual action does not necessarily imply progress towards the overall goal. In contrast, whole trajectories are often too coarse-grained and difficult to reuse across different tasks. We propose a subgoal-aware memory graph that distils past trajectories into structured, reusable evidence at the level of subgoals. We define a subgoal as consecutive actions shar- ing a single intent, the granularity at which progress toward the task becomes meaning- ful. To reduce redundancy and support cross- task reuse, we organise subgoal experiences as nodes in a graph, where related memories can be merged, extended, and retrieved based on the environment observation. On SPA-Bench, our method improves success rate by 11.11% over the baseline.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.