We already know that the entries are Gaussian with mean 0. Moreover, the columns are independent because
\(\omega _1,\ldots ,\omega _\ell \) are independent. Therefore, we focus on the covariance matrix. Let
\(1\le i,i'\le k\),
\(1\le j,j'\le \ell \), then since
\(\mathbb {E}\!\left[ \langle v_i,\omega _j\rangle \right] = 0\) we have
$$\begin{aligned} {{\,\mathrm{cov}\,}}(\langle v_i,\omega _j\rangle , \langle v_{i'},\omega _{j'}\rangle ) = \mathbb {E}\left[ \langle v_i,\omega _j\rangle \, \langle v_{i'},\omega _{j'}\rangle \right] = \mathbb {E}\left[ X_{ij}X_{i'j'}\right] , \end{aligned}$$
where
\(X_{ij} = \langle v_i,\omega _j\rangle \). Since
\(\langle v_i,\omega _j\rangle \sim \sum _{n=1}^\infty \sqrt{\lambda _n}c_n^{(j)} \langle v_i,\psi _n\rangle \), where
\(c_n^{(j)}\sim \mathcal {N}(0,1)\), we have
$$\begin{aligned} {{\,\mathrm{cov}\,}}(\langle v_i,\omega _j\rangle , \langle v_{i'},\omega _{j'}\rangle ) = \mathbb {E}\left[ \lim _{m_1,m_2\rightarrow \infty }X_{ij}^{m_1} X_{i'j'}^{m_2}\right] ,\qquad X_{ij}^{m_1}:=\sum _{n=1}^{m_1} \sqrt{\lambda _n}c_n^{(j)} \langle v_i,\psi _n\rangle . \end{aligned}$$
We first show that
\(\lim _{m_1,m_2\rightarrow \infty }\left| \mathbb {E}\!\left[ \!X_{ij}^{m_1}X_{i'j'}^{m_2}\right] -\mathbb {E}\!\left[ X_{ij}X_{i'j'}\right] \right| =0\). For any
\(m_1,m_2\ge 1\), we have by the triangle inequality,
$$\begin{aligned}&\left| \mathbb {E}\!\left[ \!X_{ij}^{m_1}X_{i'j'}^{m_2}\!\right] -\mathbb {E}\!\left[ X_{ij}X_{i'j'}\right] \right| \\&\qquad \le \mathbb {E}\!\left[ \left| X_{ij}^{m_1}X_{i'j'}^{m_2}-X_{ij}X_{i'j'}\right| \right] \\&\qquad \le \mathbb {E}\!\left[ \left| (X_{ij}^{m_1}-X_{ij})X_{i'j'}^{m_2}\right| \right] \!\!+\mathbb {E}\!\left[ \left| X_{ij}(X_{i'j'}^{m_2}-X_{i'j'})\right| \right] \\&\qquad \le \mathbb {E}\!\left[ \left| X_{ij}^{m_1}-X_{ij}\right| ^2\right] ^{\tfrac{1}{2}}\!\mathbb {E}\!\left[ \left| X_{i'j'}^{m_2}\right| ^2\right] ^{\tfrac{1}{2}}\!\! + \mathbb {E}\!\left[ \left| X_{i'j'}-X_{i'j'}^{m_2}\right| ^2\right] ^{\tfrac{1}{2}}\!\mathbb {E}\!\left[ \left| X_{ij}\right| ^2\right] ^{\tfrac{1}{2}}\!, \end{aligned}$$
where the last inequality follows from the Cauchy–Schwarz inequality. We now set out to show that both terms in the last inequality converge to zero as
\(m_1,m_2\rightarrow \infty \). The terms
\(\smash {\mathbb {E}[|X_{i'j'}^{m_2}|^2]}\) and
\(\smash {\mathbb {E}[|X_{ij}|^2]}\) are bounded by
\(\sum _{n=1}^\infty \lambda _n<\infty \), using the Cauchy–Schwarz inequality. Moreover, we have
$$\begin{aligned} \mathbb {E}\left[ \left| X_{ij}^{m_1}-X_{ij}\right| ^2\right] = \mathbb {E}\left[ \left| \sum _{n=m_1+1}^\infty \sqrt{\lambda _n}c_n^{(j)} \langle v_i,\psi _n\rangle \right| ^2\right] \le \sum _{n=m_1+1}^\infty \lambda _n \xrightarrow [m_1 \rightarrow \infty ]{} 0, \end{aligned}$$
because
\(X_{ij} - X_{ij}^{m_1}\sim \mathcal {N}(0,\sum _{n=m_1+1}^\infty \lambda _n\langle v_i,\psi _n\rangle ^2)\). Therefore, we find that
\({{\,\mathrm{cov}\,}}(X_{ij}, X_{i'j'}) = \lim _{m_1,m_2\rightarrow \infty }\mathbb {E}[X_{ij}^{m_1}X_{i'j'}^{m_2}]\) and we obtain
$$\begin{aligned} {{\,\mathrm{cov}\,}}(X_{ij},X_{i'j'})&=\lim _{m_1,m_2\rightarrow \infty }\mathbb {E}\left[ \sum _{n=1}^{m_1}\sum _{n'=1}^{m_2} \sqrt{\lambda _n \lambda _{n'}}c_n^{(j)}c_{n'}^{(j')} \langle v_i,\psi _n\rangle \langle v_{i'},\psi _{n'}\rangle \right] \\&=\lim _{m_1,m_2\rightarrow \infty }\sum _{n=1}^{m_1}\sum _{n'=1}^{m_2} \sqrt{\lambda _n \lambda _{n'}}\mathbb {E}[c_n^{(j)}c_{n'}^{(j')}] \langle v_i,\psi _n\rangle \langle v_{i'},\psi _{n'}\rangle . \end{aligned}$$
The latter expression is zero if
\(n\ne n'\) or
\(j\ne j'\) because then
\(c_n^{(j)}\) and
\(c_{n'}^{(j')}\) are independent random variables with mean 0. Since
\(\mathbb {E}[(c_n^{(j)})^2] = 1\), we have
$$\begin{aligned} {{\,\mathrm{cov}\,}}(X_{ij},X_{i'j'})= {\left\{ \begin{array}{ll} \sum _{n=1}^\infty \lambda _n \langle v_i,\psi _n\rangle \langle v_{i'},\psi _{n}\rangle , &{} j=j',\\ 0, &{} \text {otherwise}. \end{array}\right. } \end{aligned}$$
The result follows as the infinite sum is equal to the integral in Eq. (
9). To see that
\(\mathbf {C}\) is positive definite, let
\(a\in \mathbb {R}^k\), then
\(a^* \mathbf {C} a=\mathbb {E}[Z_a^2]\ge 0\), where
\(Z_a\sim \mathcal {N}(0,\sum _{n=1}^\infty \lambda _n \langle a_1 v_1+\cdots +a_k v_k,\psi _n\rangle ^2)\). Moreover,
\(a^* \mathbf {C} a = 0\) implies that
\(a=0\) because
\(v_1,\ldots ,v_k\) are orthonormal and
\(\{\psi _n\}\) is an orthonormal basis of
\(L^2(D_1)\).
\(\square \)