SABLE: Scalable Affine Bayesian Latent-space Exploration
SABLE tackles expensive optimisation in the high-dimensional latent spaces of pretrained generative models — a regime where latent dimension $D$ can reach $40{,}500$ while the evaluation budget $N$ is orders of magnitude smaller. The core idea is to use a Bayesian affine surrogate on the characteristic-radius sphere, decoupling acquisition into a data-sized in-span component and a scalar off-span norm that is prescribed by a finite-horizon schedule rather than left to expected improvement. Together with a fast recompression scheme that bounds model-refitting cost to $\mathcal{O}(M^{2}D+M^{3})$, the method achieves tens-of-millisecond decisions and strong sample efficiency on synthetic and scientific benchmarks.
SABLE is a well-motivated and empirically solid contribution to latent-space Bayesian optimisation under severe budget constraints. It correctly identifies that in the $N \ll D$ regime "without additional structural assumptions, the data cannot identify even a global linear model, let alone a flexible global surrogate," and instead builds a Bayesian affine working model on the sphere. The combination of a finite-horizon off-span exploration schedule and bounded-cost recompression yields practical decision times — tens of milliseconds versus hundreds of seconds for GP baselines — while maintaining or improving sample efficiency across most benchmarks, as the authors note: "Across synthetic benchmarks and latent-space optimisation tasks, SABLE achieves high sample efficiency and reduces decision time by several orders of magnitude at large budgets."
The strongest parts are the explicit geometric decomposition and the rigorous empirical isolation of components. Proposition 3.1 shows that after $N$ observations "the slope update has dimension at most $N-1$, yielding a decomposition of each query into an in-span component optimised by expected improvement (EI) in at most $N$ dimensions and an off-span component governed by a scalar norm." The controlled ablations in Table 1 cleanly separate the effect of hyperparameter inference (MLE vs MAP vs marginalisation) from the effect of the prescribed schedule, demonstrating that the decoupled schedule improves degree-one recovery and optimisation performance even when the underlying surrogate is unchanged, because as the authors write, "We separate these decisions because conditioning reduces uncertainty along data-informed directions, whereas directions in $\mathcal{V}_{N}^{\perp}$ retain their prior variance."
The method is built on an isotropic latent prior and a spherical search domain; while practically motivated, this is restrictive when latents are anisotropic or the generator does not concentrate mass at a characteristic radius. The paper concedes that "The work assumes an isotropic latent distribution (e.g. $\mathcal{N}(\mathbf{0}, I_D)$) and that the affine working model captures a useful signal. Objectives dominated by higher-order or quantised structure, as in the Peptides benchmark, are a weaker regime." Indeed, SABLE underperforms sep-CMA-ES on Peptides. Second, the finite-horizon schedule depends on a hand-chosen reference resolution $\lambda_{\mathrm{ref}}$ without automatic selection. Finally, the recompression stability analysis has a notable gap: "Appendix E.4 gives stability bounds for the conditional posterior and EI approximation under repeated recompression. The analysis extends to marginal EI when exact and compressed calculations share hyperparameter weights; changes caused by approximate evidence evaluation are not covered."
The compression ablation on ZonalFunctionBench (Figure 2) shows that $M=300$ tracks the uncapped representation within $0.05$ on all objectives while reducing wall-clock time per decision from tens of seconds to milliseconds, and the schedule-ablation in Table 1 verifies that prescribing the off-span norm improves recovery of the degree-one component. However, the headline claim that SABLE "reduces fitting costs by over four orders of magnitude at large budgets" is anchored to a single timing point — "At $N=D=10{,}000$, the $M=300$ variant requires 5.5 ms per decision, compared with 79.5 s for the uncapped representation used in Fan et al. (2026)" — and mixes algorithmic and implementation differences. GP baselines were truncated at 48 hours for failing to scale ("Vanilla BO, Linear BO, MSR, and BAxUS do not reach the budget within 48 hours and are reported at the evaluations completed by then"), which fairly illustrates practicality but prevents direct sample-efficiency comparisons at large $N$.
Reproducibility is partially strong but materially incomplete. Hyperparameters, discretisation grids, and prior settings are documented — "The default prior and discretisation parameters are collected in Appendix H.1" — and benchmark definitions in Appendix G.2 reference publicly available pretrained models. However, no code repository, data snapshots, or implementation checkpoints are provided in the paper or on the arXiv abstract page. The recompression procedure is algorithmically detailed in Appendix E, yet the stability guarantee omits the approximate marginal-likelihood evaluation actually used in experiments: "Appendix E.4 gives stability bounds for the conditional posterior and EI approximation under repeated recompression. The analysis extends to marginal EI when exact and compressed calculations share hyperparameter weights; changes caused by approximate evidence evaluation are not covered."
Complex objects such as engineering designs and proteins are often difficult to optimise because their design spaces exhibit high-dimensional structure and objective evaluations are expensive. Generative models provide continuous latent representations, but their latent spaces can have thousands to tens of thousands of dimensions, creating a regime where the latent dimension $D$ greatly exceeds the evaluation budget. We introduce SABLE, a Bayesian optimisation method for high-dimensional latent spaces with an isotropic distribution and limited budget. SABLE uses a Bayesian affine surrogate for prediction and acquisition, treating higher-order structure as surrogate discrepancy. After $N$ observations, the slope update has dimension at most $N-1$, yielding a decomposition of each query into an in-span component optimised by expected improvement (EI) in at most $N$ dimensions and an off-span component governed by a scalar norm. We show that the off-span norm implicitly selected by affine EI can lead to underexploration, and SABLE instead prescribes it using a finite-horizon schedule, without restricting search to a learned subspace. A recompression scheme reduces information to $\mathcal O(M)$ pseudo-observations, bounding refitting cost by $\mathcal O(M^2D+M^3)$ while keeping parameter inference, prediction, and acquisition decisions to tens of milliseconds. Across synthetic benchmarks and latent-space optimisation tasks, SABLE achieves high sample efficiency and reduces decision time by several orders of magnitude at large budgets.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.