Dave Winer

Sample-Free Expected Output Estimation for Wide MLPs via Cumulants and Hermite Expansions

This paper introduces an analytic alternative to Monte Carlo sampling for estimating the expected output of wide MLPs at initialization over Gaussian inputs. Rather than forwarding samples through the network, the method constructs approximate representations of activation distributions layer-by-layer using cumulants and Hermite expansions. For sufficiently wide networks, the approach achieves a target mean squared error at substantially lower FLOP cost than sampling, with particular advantages for rare/tail event probability estimation. The authors further demonstrate applicability to model training, suggesting a path toward reducing catastrophic tail risks in deployed models.

Failure of the large-N expansion in a Bosonic Tensor Model

A Jammed Parisi Ansatz

Deep Neural Nets as Hamiltonians

Eigenstate Condensation Phases in Finite-Dimensional Quantum Systems

Energy-constrained random quantum states in finite-dimensional Hilbert spaces exhibit eigenstate condensation, characterized by three distinct phases. These include a ground-state phase and an anti-ground-state phase, both displaying macroscopic overlap with extreme energy states. A high-temperature phase shows exponentially small overlap with individual eigenstates, distinguished by exponentially sharp phase transitions that extend its effective range.