Decoupled Marginal Sharpening for Training-Free Inference-Time Scaling
Training-free inference-time scaling aims to improve LLM reasoning without updating model weights or relying on external reward models. Existing reward-free methods sharpen the full sequence distribution, which can suppress diverse reasoning paths that support the same correct answer, and suffer from path collapse in Sequential Monte Carlo samplers. This paper proposes Decoupled Marginal Sharpening (DMS), a two-stage framework that separately controls reasoning-path concentration and answer-marginal consensus via independent exponents, paired with a novel Fully Adapted Annealed SMC sampler. The authors report the highest mean pass@1 across three base models and four benchmarks, including mathematics, science, and code tasks.
DMS presents a conceptually attractive extension of marginal sharpening by decoupling reasoning and answer concentration, backed by broad empirical gains: the highest reported mean pass@1 in all twelve model–benchmark settings. The sampling mechanics—annealed intermediate targets, predictive resampling before propagation, and island particle populations—directly address known failure modes of SMC power-sampling methods. However, the necessity of the two-stage design is weakened by an ablation where answer sharpening alone matches the full procedure.
The mathematical formulation is clean and well-motivated: reasoning sharpening encourages exploration while answer sharpening concentrates probability on answers with aggregate support across paths. The sampler innovations directly tackle path collapse, using resample-before-propagate to let copied prefixes diverge immediately, and the island variant preserves ancestor diversity on long reasoning problems. Empirically, DMS also runs faster than the Marginal Sharpening baseline despite its more complex two-stage procedure.
Practical deployment is hindered by extensive per-task and per-model hyperparameter tuning, with reasoning exponents ranging from 1.001 to 1.25 and answer exponents from 1.5 to 8 across settings (Table 3). The AIME24/25 ablation further undercuts the core claim: answer sharpening alone achieves the exact same pass@1 as the full two-stage Island-DMS, suggesting reasoning sharpening may be dispensable even on hard math tasks. Additionally, the locally fully adapted proposal requires normalizing over the full token vocabulary, yet the paper offers almost no detail on how this intractable sum is approximated in practice.
The main results show DMS leading on mean accuracy in every setting, but margins are sometimes modest and baseline variances occasionally overlap, especially on GPQA-Diamond where Marginal Sharpening reports large standard deviations. Fair comparison is complicated by differing inference budgets: DMS uses K=16 reasoning particles and M=32 answer candidates with cross-scoring, whereas Marginal Sharpening uses 32 reasoning traces and Power-SMC uses 16 full-sequence particles. While DMS reports lower wall-clock latency than Marginal Sharpening, it does not provide a standardized forward-pass count for equal-compute comparison.
The paper provides reasonably detailed hyperparameter schedules, generation budgets, and annealing checkpoints in Appendix B, which aids reproduction. However, no code repository is mentioned, and the per-task exponent grids in Appendix C imply substantial manual tuning without a principled selection protocol for new tasks. The fully adapted proposal’s vocabulary-level normalization is a critical implementation detail that is left underspecified, raising a barrier to exact independent reproduction.
We study how to improve LLM reasoning at inference time without updating model weights or using external reward models. Recent reward-free power-sampling methods define the sequence-level power target $\pi_\alpha(y\mid x)\propto p_0(y\mid x)^\alpha$, where $\alpha>1$, which increases the relative probability of completions that are already likely under the base model. Existing methods use Metropolis--Hastings or Sequential Monte Carlo (SMC) to approximately sample from this target. However, two challenges arise at different levels. At the target level, full-sequence sharpening can concentrate probability on a few exact reasoning--answer trajectories, suppressing alternative paths that support the same answer. At the sampler level, finite-particle SMC can suffer from path collapse when resampling repeatedly duplicates a few high-weight prefixes. To address these challenges, we propose Decoupled Marginal Sharpening (DMS), which separately controls how strongly the model favors particular reasoning paths and how strongly it favors final answers after aggregating support across those paths. At the target level, DMS first samples from a sharpened reasoning distribution, then marginalizes the model's answer probabilities over the resulting weighted reasoning population and sharpens the induced answer marginal. Separate exponents allow mild reasoning sharpening to encourage exploration of diverse paths, followed by stronger answer-marginal sharpening to concentrate probability on answers with greater aggregate support across those paths. At the sampler level, we develop Fully Adapted Annealed SMC, which combines an annealed sequence of power targets with locally fully adapted proposals and predictive ancestor resampling. Experiments on MATH500, GPQA-Diamond, HumanEval, and AIME24/25 across three base models show that DMS achieves the highest mean pass@1 among the compared methods in all tasks.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.