XIV Symposium on Probability and Stochastic Processes

This chapter provides an expository account of training dynamics in the Deep Linear Network (DLN) from the perspective of the geometric theory of dynamical systems. Rigorous results by several authors are unified into a thermodynamic framework for deep learning.

The analysis begins with a characterization of the invariant manifolds and Riemannian geometry in the DLN. This is followed by exact formulas for a Boltzmann entropy, as well as stochastic gradient descent of free energy using a Riemannian Langevin Equation. Several links between the DLN and other areas of mathematics are discussed, along with some open questions.

We develop an analytical approach to quantum Gaussian states in infinite-mode representation of the Canonical Commutation Relations (CCRs), using Yosida approximations to define integrability of possibly unbounded observables with respect to a state \(\rho \) (\(\rho \)-integrability). It turns out that all elements of the commutative \(*\)-algebra generated by a possibly unbounded \(\rho \)-integrable observable A, denoted by \(\langle A\rangle \), are normal and \(\rho \)-integrable. Besides, \(\langle A\rangle \) can be endowed with the well-defined norm \(\|\cdot \|_\rho := {\mathrm {tr}}\left (\rho |\cdot | \right )\). Our approach allows us to rigorously establish fundamental properties and derive key formulae for the mean value vector and the covariance operator. We additionally show that the covariance operator S of any Gaussian state is real, bounded, positive, and invertible, with the property that \(S-iJ\geq 0\), being J the multiplication operator by \(-i\) on \(\ell _2({\mathbb N})\).

This chapter concerns two-person Markov games in Borel spaces with additive transition and additive rewards. The existence of stationary Nash equilibria is proven for rewards that are not necessarily bounded, under certain continuity and compactness assumptions. We also show that players can choose equilibrium strategies concentrated on at most two actions at each state.