This paper studies nonlinear in-context learning via single-index models, comparing a kernel learner (fixed spherical-harmonic features followed by linear attention) against a feature learner (linear attention on raw inputs followed by a learned nonlinear readout). Using the replica method, the authors derive predictions for memorization and generalization errors across varying pretraining data, task diversity, and context lengths, and validate them numerically over a broad parameter range. The analysis reveals phase diagrams for architectural advantage and qualitatively different context-length scalings: the kernel learner needs $L = \Theta(d^k)$ to recover degree-$k$ harmonics, while the feature learner exploits all teacher components at $L = \Theta(d)$.

This paper develops a high-dimensional theory for why masked autoencoders learn better representations than standard autoencoders. In a spiked model with heterogeneous noise, the authors prove that a linear masked autoencoder recovers the latent feature at linear sample complexity $n=O(d)$ where PCA fails. They further quantify how dynamic mask resampling lowers sample complexity compared to static masking, and validate the qualitative predictions in CNNs, vision transformers, and a BERT pilot.

Disorder is ubiquitous in functional crystals—nearly half of ICSD entries show some form of disorder—yet most generative models assume perfectly ordered lattices with deterministic site occupations. EP-Flow attacks formula-conditioned disordered crystal structure prediction without site-level annotations by representing occupancies as an Occupancy Distribution Matrix (ODM) that lives on a formula-dependent transportation polytope. It proposes a flow-matching model that canonicalizes heterogeneous polytopes into a shared double-centered space via log-domain double centering, learns a marginal-preserving velocity field, and recovers valid occupancies with a differentiable Sinkhorn inverse map, jointly generating fractional coordinates and lattice parameters. If the absolute performance can be scaled, this formalism could become a standard way to handle substitutional, vacancy, and interstitial disorder in generative materials design.

Recent theoretical models of diffusion as coupled Ornstein-Uhlenbeck processes predict a hierarchy of interaction timescales creating a synchronization gap between global and local committing modes. This work investigates how this gap mechanistically emerges within pretrained Diffusion Transformers by introducing a controlled architectural realization of replica coupling via symmetric cross-attention gates with strength $g$. Through linearized analysis and empirical probing of DiT-XL/2 across all 28 layers, the authors demonstrate that the gap is an intrinsic, depth-localized property that collapses under strong coupling as $\mathcal{O}(\frac{1-g}{1+g})$, providing a bridge between continuous statistical physics and discrete transformer dynamics.