The minimal width of neural networks and immersions of real projective space

stat.ML cs.LG BaoZhi Chu, Ben Weinkove · Oct 14, 2026

What it does

Why it matters

This paper attacks the minimal-width problem by importing tools from differential topology, proving that whenever real projective space $\mathbb{RP}^{n-1}$ smoothly immerses into $\mathbb{R}^n$, any network with a continuous monotone...

Main concern

The paper introduces an elegant geometric-topological framework for proving lower bounds on neural network width. Theorem 1.

Community signal

0 up · 0 down

AI Review AI reviewed

Plain-language introduction

What is the least width required for a deep neural network to universally approximate continuous functions from $\mathbb{R}^n$ to $\mathbb{R}^m$? This paper attacks the minimal-width problem by importing tools from differential topology, proving that whenever real projective space $\mathbb{RP}^{n-1}$ smoothly immerses into $\mathbb{R}^n$, any network with a continuous monotone activation must have width at least $n+\min\{n,m\}$ to approximate arbitrary maps. This yields the exact ReLU minimal width $n+m$ for the special dimensions $n\in\{2,3,4,7,8\}$ with $m\le n$, and exposes a surprising link between universal approximation and the immersibility of projective spaces.

Critical review

Verdict

Bottom line

The paper introduces an elegant geometric-topological framework for proving lower bounds on neural network width. Theorem 1.1 is cleanly stated and rigorously proved, and Corollary 1.1 gives exact constants for ReLU networks in the dimensions where $\mathbb{RP}^{n-1}$ immerses in $\mathbb{R}^n$. The novelty and technical sophistication are high, but the practical scope is narrow because the topological hypothesis holds only for $n\in\{2,3,4,7,8\}$, leaving the general minimal-width problem unsettled.

“Assume the activation function $\sigma$ is continuous and monotone. Let $n\geq 2$ and $m\geq 1$. Assume that there is a smooth immersion $i:\mathbb{RP}^{n-1}\to\mathbb{R}^{n}$. Then ... $w_{\mathrm{min}}(n,m)\geq n+\min\{n,m\}$.”

Chu and Weinkove · Theorem 1.1

“For the ReLU activation, $w_{\min}(n,m)=n+m$ if $n\in\{2,3,4,7,8\}$ and $1\leq m\leq n$.”

Chu and Weinkove · Corollary 1.1

What holds up

The core topological argument is compelling and well-executed. The persistence-of-zeros theorem for odd homotopies (Theorem 2.1) provides a solid foundation, and the construction of the non-approximable map $f$ in Section 3 carefully handles orientation issues for both even and odd $n$ via antipodal quotients and reflections. Lemma 5.1, which shows that any sufficiently narrow network can be approximated by an injective map followed by a projection, is adapted effectively to turn the topological obstruction into a width lower bound.

“If $g:A\to\mathbb{R}^{n}$ and $h:A\to\mathbb{R}^{n-1}$ are smooth maps, and $g$ satisfies $\sup_{A}|f-g|\leq\varepsilon$, then $G:=(g,h)$ is not injective.”

Chu and Weinkove · Theorem 3.1

“For every nonempty compact $K\subset\mathbb{R}^{n}$ and every $\varepsilon>0$, there is a smooth injective map $G:\mathbb{R}^{n}\to\mathbb{R}^{N}$ such that $\sup_{K}|\pi_{N,m}\circ G-F|<\varepsilon$.”

Chu and Weinkove · Lemma 5.1

Main concerns

The biggest limitation is dimensional: the sharp lower bound is conditional on the immersibility of $\mathbb{RP}^{n-1}$ into $\mathbb{R}^n$, which by classical topology is known only for $n\in\{2,3,4,7,8\}$. For all other input dimensions the paper gives no improvement over existing weaker bounds such as $\max\{n,m\}+1$.\n\nAdditionally, the activation must be continuous and monotone, excluding common non-monotone activations like GELU or SiLU. The authors are admirably candid that the formula $w_{\min}=n+m$ is not expected universally, noting explicit regimes where it already fails.

“We do not expect that the formula $w_{\text{min}}(n,m)=n+m$ holds for all values of $n,m$ with $1\leq m\leq n$, although we cannot yet provide a counterexample. On the other hand, it is known that if $m>2n\geq 4$ then $w_{\text{min}}(n,m)=n+m$ fails [8, Theorem 4.8].”

Chu and Weinkove · Section 1, final remarks

Evidence and comparison

The upper bound $w_{\min}\le n+m$ from Hanin and Sellke is correctly invoked and matches the new lower bound in the restricted setting, yielding equality. Comparisons to Hwang's $n=m=2$ result and to Kim-Min-Park are fair and accurately position the contribution. The topological evidence—mod-2 intersection theory and Hirsch's immersion classification—is classical and well-documented. Because the paper is purely theoretical, no empirical validation is required or provided.

“For every $d_{in},d_{out}\geq 1$, $d_{in}+1\leq w_{\text{min}}(d_{in},d_{out})\leq d_{in}+d_{out}.”

Hanin and Sellke, arXiv:1710.11278v2 · Theorem 1

“Combining this bound with (1.1) and a well-known classification of the dimensions $n$ for which $\mathbb{RP}^{n-1}$ admits a smooth immersion into $\mathbb{R}^{n}$, we obtain the following.”

Chu and Weinkove · Section 1, after Theorem 1.1

Reproducibility

Reproducing the results requires no code or data, but demands substantial expertise in differential topology. The authors provide a completely explicit construction for the $n=2$ case in Section 4, which aids understanding, and the general case in Section 3 is written out in coordinates with orientation checks. All hyperparameters and constants are existential ($\varepsilon_0$, $\delta$), as is standard for topology-based lower bounds. A potential obstacle for readers is the reliance on mod-2 intersection theory; while the authors cite Hirsch's textbook, a short appendix summarizing the needed background would improve accessibility.

“By the definition of the width, we must have $N\geq\max\{n,m\}$. We may therefore apply Lemma 5.1, with $K=A$, to obtain a smooth injective map $G:\mathbb{R}^{n}\to\mathbb{R}^{N}$ such that $\sup_{A}|\pi_{N,m}\circ G-F|<\varepsilon_{0}$.”

Chu and Weinkove · Section 5

Abstract

We investigate the minimal width of deep neural networks that can approximate all continuous functions from $\mathbb{R}^n$ to $\mathbb{R}^m$ in the topology of uniform convergence on compact subsets. We show that whenever the real projective space $\mathbb{RP}^{n-1}$ admits a smooth immersion into $\mathbb{R}^n$, the minimal width is at least $n+\min\{n,m\}$, assuming that the activation function is continuous and monotone. As a consequence, the minimal width of ReLU networks is $n+m$ whenever $n$ is equal to 2, 3, 4, 7 or 8 and $1\le m \le n$, extending results of Hanin-Sellke and Hwang. This suggests an intriguing connection between the approximation theory of neural networks and differential topology.

Challenge the Review

Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.

Challenges are public to read, but only signed-in members can post them. Your challenge text is stored with your account for moderation, but usernames are not shown in the public thread.

No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.