In the infinite-width limit, deep neural networks induce isotropic Gaussian fields whose covariance encodes information about the architecture and the activation function.

This talk presents a unified framework, based on three recent works, showing that spectral and geometric descriptors of random networks exhibit the same three-regime classification.

In [1], we introduce spectral complexity and classify activation functions into sparse, low-disorder, and high-disorder regimes, according to the asymptotic behavior of the angular power spectrum. This reveals structural differences in network expressivity, with sparsity emerging prominently in deep ReLU architectures.

In [2], we study level-set boundaries. For non-smooth activations, such as the Heaviside function, these boundaries may exhibit fractal behavior, with Hausdorff dimension increasing with depth. For smoother activations, the boundary volume displays contraction, stability, or exponential growth, matching the spectral regimes.

In [3], we analyze critical points of the limiting fields. Under suitable regularity assumptions, we obtain asymptotic formulas for their expected number, at fixed index or above a threshold. These formulas reveal the same trichotomy: convergence, polynomial growth, or exponential proliferation.

Overall, these results show that the trichotomy is universal and governed by the local behavior of the covariance kernel near its fixed points.

[1] Di Lillo, S., Marinucci, D., Salvi, M., Vigogna, S.: Spectral complexity of deep neural networks. SIAM Journal on Mathematics of Data Science 7(3), 1154-1183 (2025). DOI: 10.1137/24M1675746

[2] Di Lillo, S., Marinucci, D., Salvi, M., Vigogna, S.: Fractal and regular geometry of deep neural networks. arXiv: 2504.06250

[3] Di Lillo, S.: Critical points of random neural networks. arXiv: 2505.17000