Latent Preference Modeling
Abstract
Users often omit essential details in their requests to LLM-based agents, resulting in under-specified inputs for tool use. This poses a fundamental challenge for tool-augmented agents, as API execution typically requires complete arguments, highlighting the need for personalized tool calling. To study this problem in a more realistic setup, we present Multi-Session Personalized Tool Calling (MPT), a benchmark comprising 4,695 instances over 459 multi-session interaction histories that cover three challenges: Preference Recall, Induction, and Transfer. We further propose PRefine, a test-time memory method that maintains the user's latent preference as a textual hypothesis revised through a generate–verify–refine loop. Across five LLMs, existing memory systems underperform full-history prompting; PRefine outperforms all baselines and alone surpasses it on Preference Transfer. These results indicate that memory for personalized agents must abstract behavior into preferences, rather than simply archive it.
Overview
Motivation
Personalized tool-using agents must often act on under-specified requests. A query as simple as "Book a flight for my trip" specifies neither the origin, the destination, nor the price. Prior work has largely assumed that user preferences are provided explicitly—as standing instructions, curated profiles, or arguments that appeared earlier in the context—reducing personalized tool calling to a search problem. We argue that user preferences are, in reality, latent, and thus should be inferred rather than retrieved: the cues are mostly implicit in the user's past tool calls.
Research Focus
Consider a user who books a cheap restaurant, tours a free attraction, and rents a compact car—each choice triggering a distinct tool with no shared parameters. To infer that this user favors affordable options, a preference never stated, the agent must generalize across these seemingly unrelated calls. We study whether agents can first model such a latent preference from past behavior, then reason from it to an executable argument value in the target function—a cross-function correspondence rather than a stored-value lookup.
| Task Requirement | Challenge Dimensions | |||
|---|---|---|---|---|
| Benchmark | Implicit Interaction | Personalized API Arguments | Preference Reasoning | Quantified Interference |
| NLSI | ✗ | ✓ | ✗ | ✗ |
| PTBench | ✗ | ✓ | ✗ | ✗ |
| PEToolBench | ✓ | ✓ | ✗ | ✗ |
| PrefEval | ✓ | ✗ | ✗ | ✓ |
| CUPID | ✓ | ✗ | ✗ | ✗ |
| PersonaMem | ✓ | ✗ | ✗ | ✗ |
| MPT (ours) | ✓ | ✓ | ✓ | ✓ |
MPT uniquely combines implicit interaction, personalized API arguments, preference reasoning, and quantified interference.
Benchmark: MPT
MPT (Multi-Session Personalized Tool Calling) comprises 4,695 instances over 459 multi-session interaction histories, totaling 6,961 sessions and 95,523 turns (on average, 15.2 sessions per dialogue and 13.7 turns per session), composed from three source corpora (SGD, DuRecDial 2.0, and HomeBench) with five latent preferences realized over 16 function–parameter pairs across 13 functions. Instances split into three reasoning types by how much evidence prior sessions provide for the target argument: Preference Recall (1,375; the argument recurs, so direct reuse suffices), Preference Induction (1,944; a single mention must be corroborated by cross-function evidence), and Preference Transfer (1,376; the argument never occurs and the preference must be generalized to it). Each query comes in context-guided and context-free variants.
| Preference | Function(parameter = argument) |
|---|---|
| low_cost | GetRestaurants(price_range = cheap) |
GetRentalCars(car_type = Compact) | |
GetHotels(average_star = 1,2) | |
GetRideSharing(shared_ride = True) | |
GetTravel(free_entry = True) | |
GetFlights(flight_class = Economy) | |
| high_cost | GetRentalCars(car_type = Full-size) |
GetHotels(average_star = 4,5) | |
| prefers_star | GetMovies(starring = ⟨celebrity⟩) |
GetMusic(artist = ⟨celebrity⟩) | |
| eco | GetAirConditioner(eco_mode = on) |
GetHeating(eco_mode = on) | |
GetWaterHeater(eco_mode = on) | |
| solo_usage | GetBuses(group_size = 1) |
GetFlights(passengers = 1) | |
GetRideSharing(number_of_seats = 1) | |
GetEvents(number_of_tickets = 1) | |
GetRestaurants(number_of_seats = 1) |
⟨celebrity⟩ ranges over 20 identities mined from DuRecDial 2.0; the preferred identity varies across users.
Method: PRefine
only verified hypotheses enter memory.
PRefine maintains the estimate of the user's latent preference as a textual hypothesis revised across sessions. After each session, a generator proposes a preference hypothesis from the previous memory and the session's tool calls; a verifier evaluates it against the accumulated history for (1) Evidence Support, (2) Abstraction Quality, (3) Actionability, and (4) Temporal Consistency; rejected candidates are refined with verifier feedback for up to K = 5 rounds. A hypothesis enters memory only after being verified, yielding a memory that is an abstraction of user behavior rather than a raw log.
Insufficient evidence and over-specificity account for 64.4% of rejections.
Main Results
We evaluate five LLMs (GPT-5, Claude Haiku 4.5, GPT-OSS-20B, Gemma4-12B, and Qwen3-8B) combined with six methods, reporting Overall-F1 along with Explicit-F1 (extracting stated arguments) and Implicit-F1 (completing omitted, preference-sensitive ones). Agents reliably recover explicit arguments but struggle with omitted ones, and performance degrades from Recall through Induction to Transfer. Retrieval- and organization-based memories (RAG, Mem0, LangMem) even fall below full-history prompting on context-free queries, indicating that surfacing past records is not enough unless the evidence is abstracted into a preference. PRefine outperforms all baselines (O-F1 73.57 context-guided / 60.55 context-free; +15.39 / +25.26 over full-history prompting in the five-model macro average) and is the only method to exceed full-history prompting on Preference Transfer.
| Context-Guided Query (qcg) | Context-Free Query (qcf) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Method | Recall | Induction | Transfer | Avg. | Δ | Recall | Induction | Transfer | Avg. | Δ |
| Full-History Prompting | 64.85 | 63.00 | 46.67 | 58.17 | – | 49.19 | 39.16 | 17.51 | 35.29 | – |
| RAG | 64.75 | 61.07 | 56.98 | 60.93 | +2.76 | 31.41 | 30.41 | 15.21 | 25.67 | −9.61 |
| Mem0 | 64.66 | 62.40 | 56.30 | 61.12 | +2.94 | 40.78 | 37.32 | 15.66 | 31.25 | −4.04 |
| LangMem | 66.19 | 64.12 | 55.46 | 61.92 | +3.75 | 44.55 | 42.96 | 16.51 | 34.67 | −0.61 |
| A-MEM | 67.87 | 63.59 | 58.48 | 63.31 | +5.14 | 63.99 | 55.58 | 13.42 | 44.33 | +9.04 |
| PRefine (Ours) | 80.53 | 76.11 | 64.06 | 73.57 | +15.39 | 77.64 | 71.69 | 32.31 | 60.55 | +25.26 |
EXP-F1 remains high across methods, whereas IMP-F1 declines sharply on Preference Transfer.
Analysis: Evidence Interference
We quantify how hard a history makes preference modeling with a new evidence interference score: each observation is classified as supporting, competing, or unrelated to the target preference, and their weighted ratio stratifies instances by difficulty. All methods degrade as interference increases, but PRefine remains the most robust across all quintiles; isolating competing evidence shows it stays best in every contested subset. Ablations further show that verification—not repeated refinement—drives the gains: unverified refinement degrades below full-history prompting, while the full generate–verify–refine loop peaks at K = 5.
Performance declines as interference increases; PRefine remains the most robust.
PRefine performs best across all three axes.
| Method | Generate | Verify | Refine | Context-Guided (qcg) | Context-Free (qcf) | Overall |
|---|---|---|---|---|---|---|
| Full-History Prompting | – | – | – | 51.99 | 29.66 | 44.82 |
| Generator-only | ✓ | – | – | 51.17 | 21.51 | 43.81 |
| Generator–Refiner | ✓ | – | K=1 | 50.53 | 14.29 | 42.34 |
| K=3 | 49.90 | 11.96 | 41.46 | |||
| K=5 | 50.37 | 13.43 | 42.09 | |||
| Generator–Verifier–Refiner | ✓ | ✓ | K=1 | 63.78 | 55.70 | 61.47 |
| K=3 | 66.08 | 57.34 | 63.49 | |||
| K=5 | 66.45 | 57.94 | 63.95 | |||
| K=10 | 63.71 | 57.05 | 61.84 |
Generation alone does not beat full-history prompting and unverified refinement degrades further; the verifier drives the gains, peaking at K = 5.
Efficiency
Because PRefine revises a compact preference-level hypothesis rather than accumulating session-level records, it exposes an average of only 26.36 memory tokens per test instance—versus 96.88 for RAG, 121.46 for Mem0, 259.54 for LangMem, and 1,082.58 for A-MEM—and its memory store stays nearly constant as sessions accumulate while the baselines grow continuously. Memory construction costs 8,855.57 tokens per session despite the refinement loop: 6.0% fewer than Mem0 and 67.5% fewer than A-MEM.
| Method | LLM Input / Session | Embedding / Session | Output / Session | Total / Session |
|---|---|---|---|---|
| RAG | 0.00 | 237.04 | 0.00 | 237.04 |
| Mem0 | 8,382.97 | 520.57 | 520.57 | 9,424.11 |
| LangMem | 2,088.63 | 343.94 | 343.70 | 2,776.27 |
| A-MEM | 16,354.19 | 1,212.50 | 9,709.97 | 27,276.66 |
| PRefine | 8,814.03 | 0.00 | 41.54 | 8,855.57 |
PRefine (generate–verify–refine, K = 5) costs 6.0% less than Mem0 and 67.5% less than A-MEM.
PRefine keeps both the smallest context and a nearly constant store.
Key Findings
Memory for personalized agents must abstract behavior into preferences, rather than simply archive it. Surfacing past records—whether by retrieval or by organized notes—is not enough; hypotheses refined through verification are what enable cross-function generalization. Transfer under high interference remains challenging, calling for memory methods that better resolve competing evidence and ground inferred preferences in unseen API parameters.
BibTeX
@article{yoon2026latent,
title = {Latent Preference Modeling for Multi-Session Personalized Tool Calling},
author = {Yoon, Yejin and Kim, Minseo and Kim, Taeuk},
journal = {arXiv preprint arXiv:2604.17886},
year = {2026}
}