DIAL: Steerable Language Models with Distilled Instruction-Adherence Levels
DIAL tackles the challenge of fine-grained control over instruction adherence—how strongly an LLM should follow instructions like "Keep it short." The core idea is to distill classifier-free guidance (CFG) into a lightweight, text-conditioned steering adapter that edits hidden states at a single layer. This matters because it promises scalable, efficient, adjustable adherence across diverse instructions without requiring labeled responses or per-instruction fitting.
DIAL presents an elegant combination of on-policy CFG distillation and adaptive hidden-state steering. The paper demonstrates monotonic control over concept and persona expression across six LLM families with a 20.6M-parameter adapter that retains 96–99% of base-LLM throughput. However, its effectiveness is uneven: verifiable instruction following lags behind prompting when steering alone, and high adherence levels still risk coherence degradation. The practical operating range is also bounded to $\alpha \in [0.5,2]$ due to CFG degradation beyond $\alpha=2$.
The fine-grained control claim is strongly supported. On AxBench, DIAL smoothly increases concept expression across adherence levels and achieves better overall scores than text-conditioned baselines like FLAS, despite using only instruction text rather than labeled completions. Inference efficiency is convincingly demonstrated in vLLM, where DIAL’s per-token latency matches the base LLM and throughput far exceeds that of CFG.
The primary concern is verifiable instruction following. On IFEval, DIAL-IF alone achieves only 53.8% strict accuracy on Gemma-2-9B at $\alpha=2$, far below prompting at 72.8% (Table 4). While prompt+DIAL-IF is competitive with prompting on some models, it underperforms on others, and the paper concedes that steering alone is insufficient for some instruction types. Additionally, the evaluation relies on LLM judges for concepts and personas, and the fine-grained range is limited because CFG coherence degrades beyond $\alpha=2$.
Evidence supports concept-steering and efficiency comparisons well. DIAL outperforms FLAS on AxBench overall scores (1.059 vs. 0.997 Held-In) without requiring labeled responses (Section 4.1). The comparison to per-concept steering methods is fair because DIAL uses a single shared adapter. For verifiable instructions, the evidence is weaker because IFEval’s binary checks cannot capture intermediate adherence, and the gains from combining steering with prompting are inconsistent across LLM families (Section 4.3).
The paper reports adapter architecture (20.6M parameters, 512-dim features, memory window of 16), training setup (LogUniform $\alpha \sim [0.5,2]$, JSD loss), and hardware (H200), with full hyperparameters in Appendix B. The SteerIF dataset is introduced and described. However, the main text does not mention public code release, and exact random seeds and per-LLM layer selection criteria are not fully detailed in the provided text, which would impede independent reproduction.
When a user says "Keep it short", how short should the LLM's response be? More generally, users and applications require different levels of instruction adherence, but prompting does not reliably provide fine-grained control over them. We therefore study "steerable language models" that provide fine-grained numerical control over the degree to which models follow general instructions specified in text. However, existing approaches do not jointly offer fine-grained control, efficiency, and scalability across diverse instructions. To address this gap, we introduce DIAL (Distilled Instruction-Adherence Levels) to achieve all three properties. For fine-grained control, we perform classifier-free guidance (CFG) with a frozen LLM to provide training signals for on-policy distillation of instruction adherence at different levels. For efficiency, we design a lightweight steering adapter that uses a classifier over the LLM's hidden states to compute a closed-form steering edit. The classifier adaptively sets the steering magnitude at each token based on the target adherence level, while accounting for the geometry of the LLM's hidden states to preserve response coherence. For scalability, the classifier is text-conditioned, allowing a shared adapter to support new instructions without per-instruction fitting, and training requires only instruction and request text. Evaluation across multiple LLM families shows that DIAL achieves a competitive balance between concept expression and response fluency, with concept scores increasing monotonically across adherence levels. We introduce SteerIF, an instruction-only dataset of 98, 625 concept, persona, verifiable, and descriptive instructions. When trained on SteerIF, DIAL provides fine-grained control over personas and, when combined with prompting, improves verifiable instruction following.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.