Self-Conditioning Enables Deep Sequential Reasoning in Flow Matching
The paper tackles whether iterative image generation with flow matching can actually perform deep sequential reasoning, or whether the many sampling steps are merely an illusion of compute. It proves that vanilla flow matching collapses to constant-depth circuits because each ODE step changes state by only $O(1/K)$, but shows that self-conditioning—feeding back an endpoint prediction as an internal scratchpad—breaks this barrier and can reach $\mathsf{P/poly}$ and even $\mathsf{PSPACE/poly}$. The result is a sharp theoretical and empirical argument that how information flows between sampling steps matters more than how many steps you take.
The paper delivers an elegant circuit-complexity separation that is novel and mostly well supported: vanilla flow matching is confined to the backbone's circuit class, while self-conditioning lifts expressive power to space-bounded computation. The synthetic experiments cleanly validate the predicted NFE-scaling behavior. Still, the practical impact is constrained by strong regularity assumptions, backbone choices with 2026 commercial model cards, and the fact that the $\mathsf{TC}^0 \neq \mathsf{NC}^1$ separation needed for some claims remains a conjecture.
The theoretical mechanism is compelling and clearly communicated. The parallel Picard iteration proof (Lemma 1) is a genuine technical contribution: it shows the vanilla sampler's $K$ Euler steps can be compressed to $O(L)$ rounds because the $O(1/K)$ increments let errors decay factorially. The empirical design is equally sharp—the authors construct three synthetic tasks whose known circuit complexities (NC1-complete line tracing, TC0-complete long multiplication and text rendering) sit exactly on the predicted barrier, and the NFE-scaling curves match the theory almost perfectly.
The 'mild regularity conditions' are not mild in practice. Theorem 1 demands a constant Lipschitz constant $L$ and bounded precision, yet the empirical estimate is $L \approx 400$, and §3.4 notes the $O(L)$ bound is tight—so the theorem would fail for realistic Lipschitz growth. The training comparison is also debatable: to match FLOPs, SelfCondFM trains on fewer unique examples because it makes two forward passes per sample (one with $z_0=\mathbf{0}^n$ and one with the predicted endpoint). That SelfCondFM still wins is notable, but the design choice disadvantages it and the paper frames it as a controlled match rather than an inherent win.
The complexity separation for TC0 backbones rests on the unproven conjecture $\mathsf{TC}^0 \neq \mathsf{NC}^1$ (acknowledged in §3.3), and the $\mathsf{PSPACE/poly}$ claim requires up to exponentially many steps—far beyond any practical sampler. The broader DiffThinker evaluation (Table 2) switches from FLUX.2-klein-base-4B to Qwen-Image-Edit-2511, so the strong results confound backbone scale with the self-conditioning mechanism.
The evidence supports the core expressivity separation on the three synthetic tasks, where circuit complexity is known and the performance gaps are dramatic (e.g., long text rendering stays near zero for VanillaFM while SelfCondFM rises with compute). However, the three tasks are all structured text-in-image problems, so the claim of broader 'visual reasoning' is not fully stress-tested on natural images. Comparisons to EndoCoT in Table 2 are misleading because EndoCoT uses a multimodal LLM with latent chain-of-thought, a fundamentally different architecture rather than a conditioning tweak on the same flow backbone. The positioning against prior self-conditioning work (Chen et al., 2022) is fair but the authors could have compared more directly against other scratchpad mechanisms, such as explicit latent memory or iterative refinement with skip connections.
Code is released on GitHub and the appendices provide proof sketches, hyperparameters, and dataset descriptions, which is commendable. Full reproduction is nonetheless blocked by proprietary backbones: FLUX.2-klein-base-4B and Qwen-Image-Edit-2511 are commercial weights that may not be openly available. The paper also discloses substantial AI assistance in proofs, dataset generation, and code implementation, with multi-author review. That process is transparent but adds layers between the reported results and independent verification. Exact random seeds, data generation prompts, and evaluation scripts are referenced but not shown in the main text or appendices provided.
Recent work has explored visual reasoning through image generation, where models solve visual-reasoning problems by directly generating solution images. Intuitively, iterative samplers such as flow matching are well suited to this task, since each sampling step may resemble one reasoning step. However, in this work, from a circuit complexity perspective, we show that this intuition is misleading. We prove that as long as the model outputs satisfy certain mild regularity conditions, the trajectory of the vanilla flow-matching sampler can be simulated by a constant number of sequential rounds of parallel model evaluations. Consequently, additional sampling steps cannot increase the expressive power of a constant-depth backbone model under the standard complexity-theoretic model of transformers. We then reveal that self-conditioning, which conditions the model on the previous step's estimate of the final image, plays a previously unrecognized role in removing this bottleneck. Controlled experiments on diverse visual-reasoning tasks show that flow matching with self-conditioning achieves higher accuracy and greater reasoning depth than vanilla flow matching, and its performance improves with more sampling steps. Theoretically, we prove that with self-conditioning, a constant-depth backbone can achieve expressive power equivalent to that of space-bounded Turing machines. Our study suggests a design principle for future visual reasoning research: carry more information between sampling steps, so that additional steps translate into deeper reasoning.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.