Learning What to Ask for Tomorrow: Active Preference Acquisition for Long-Term Personalized Web Agents
Persistent web agents improve over repeated tasks only if they know user preferences that history alone cannot reveal. This paper formalizes Active Preference Acquisition (APA), where an agent decides which missing attributes to ask about by estimating their Prospective Personalization Value: the expected cumulative utility gain across future tasks minus asking and privacy costs. The core insight is that many preferences are worth acquiring only when reused across many later tasks, since no single booking or purchase may justify the cost of asking.
The paper makes a precise conceptual contribution: it isolates an amortization regime in which myopic, current-task methods systematically fail, and provides both a formal criterion and a benchmark to measure it. The evaluation is rigorous within its synthetic sandbox, but entirely limited to it. As the authors note, "APA recovers 84.2% of its utility gain with 27.6% of its questions" (Abstract), yet the underlying claim that "a criterion valuing only the current task never acquires a preference whose benefit on every single task is below the asking cost, however often it is reused" (Proposition 3.1) has so far only been demonstrated with simulated users and exact utilities.
The distinction between current-task value-of-information and cross-task prospective value is crisply formalized in $PPV_T(m) = \mathbb{E}[\sum_{k=t+1}^{T} \Delta U(\tau_k; m)] - C_{\lambda}^{\beta}(m)$, and Proposition 3.1 gives a clean characterization of when amortization alone can justify acquisition. The controlled benchmark ActiveWebPref is a genuine asset: because hidden profiles and exact utilities are known, the counterfactual value of every unasked preference is measurable without LLM judges. The headline result is striking: "At a horizon of 20 tasks, 34.4% of budget-feasible valuable missing preferences are amortization-dependent; APA acquires 80.3% of them, whereas current-task value of information acquires 5.1%" (Section 6.2).
The entire empirical argument rests on a synthetic benchmark with deterministic simulated users, fixed attribute schemas, and a single moderately-sized LLM backbone (Qwen3-8B). The authors candidly acknowledge that "real users may answer vaguely, change their minds, or find questions annoying in ways that our cost model only approximates" (Section 8), but no human evaluation or live-web validation is provided to bound this gap. Additionally, one of APA's key modeling components—forecasting future task-family distributions—shows no measurable benefit: "Replacing the learned distribution over future task families with a uniform prior leaves NU essentially unchanged" (Section 7). This suggests the benchmark's regular structure may not stress-test the method's intended advantage, and leaves open whether the full framework would survive richer, more heterogeneous task distributions.
Within ActiveWebPref the evidence supports the claims: the shared deterministic task solver isolates the acquisition decision, and the oracle comparisons establish an upper bound. However, the paper does not empirically compare against end-to-end memory or personalization baselines on any established web-agent benchmark, arguing instead that existing benchmarks "do not directly provide a protocol with hidden but queryable profiles and counterfactual utilities of unasked preferences" (Section 5). While this distinction is conceptually fair, it leaves unanswered whether APA improves real-world agent performance or merely solves a carefully constructed acquisition game.
The authors publish code, data, and prompts, and report that "All model outputs are cached, and every table and figure is generated from the logged results" (Section 5.3), which aids exact reproduction. Still, independent replication would require re-running Qwen3-8B at temperature 0 with the same prompt suite, and the paper omits compute budgets, wall-clock times, and the size of the cache. Because the benchmark is fully synthetic and disconnected from live websites or human users, reproducing the numbers is feasible while validating real-world transfer is not.
Web agents increasingly serve the same user across repeated online tasks such as booking travel, shopping, and scheduling, so their long-term usefulness depends on how well they know the user's preferences. Passive memory and personalization methods cannot obtain preferences the user never reveals, and clarification methods that value a question by its benefit to the current task overlook the benefit of reusing the answer in later tasks. We study Active Preference Acquisition (APA): which missing preferences are worth the cost of asking? We define the Prospective Personalization Value (PPV) of a missing preference as its expected cumulative utility gain over future tasks minus the effort and disclosure-risk cost of asking, and select questions under a per-interaction budget. We characterize when cumulative reuse justifies acquisition despite insufficient single-task benefit, and evaluate this regime with an initial-profile diagnostic. APA computes the counterfactual gain of each candidate preference on public task prototypes with the agent's own task solver and aggregates it by the expected reuse of each task family. We evaluate it on ActiveWebPref, a controlled longitudinal benchmark with hidden but queryable user profiles and exact task utilities. At a horizon of 20 tasks, 34.4% of budget-feasible valuable missing preferences are amortization-dependent; APA acquires 80.3% of them, whereas current-task value of information acquires 5.1%. Under per-user asking-cost caps matched to APA's realized expenditure, APA achieves higher net utility than four heuristic selection baselines. In our cost setting, asking about every unknown preference yields lower net utility than not asking, while APA recovers 84.2% of its utility gain with 27.6% of its questions. The value of acquiring a user preference should account for its reuse across future tasks, not only its benefit to the task at hand.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.