Functional linear model

doi:10.1016/S0167-7152(99)00036-X

Statistics & Probability Letters

Volume 45, Issue 1, 15 October 1999, Pages 11-22

https://doi.org/10.1016/S0167-7152(99)00036-X Get rights and content

Abstract

In this paper, we study a regression model in which explanatory variables are sampling points of a continuous-time process. We propose an estimator of regression by means of a Functional Principal Component Analysis analogous to the one introduced by Bosq [(1991) NATO, ASI Series, pp. 509–529] in the case of Hilbertian AR processes. Both convergence in probability and almost sure convergence of this estimator are stated.

Introduction

Classical regression models, such as generalized linear models, may be inadequate in some statistical studies: it is the case when explanatory variables are digitized points of a curve. Examples can be found in different fields of application such as chemometrics (Frank and Friedman, 1993), linguistic (Hastie et al., 1995) and many other areas (see Hastie and Mallows, 1993; Ramsay and Silverman, 1997, among others).

In this context, Frank and Friedman (1993) describe and compare different estimation procedures – Partial Least Squares, Ridge Regression and Principal Component Regression – which take into account both the number of explanatory variables (which may exceed the sample size) and the high correlation between these variables. On the other hand, several authors (see below) have developped models which allow to describe the “functional” nature of explanatory variables.

Formally, the above situation can be described through the following functional linear model. Let Y be a real random variable (r.r.v.) and X=(X(t),t∈[0,1]) be a continuous-time process defined on the same space $(Ω, A,P)$ . Assuming that $E (∫ 01 X^{2} (t) d t)<∞$ , the dependence between X and Y is expressed as $Y= ∫ 01 ψ(t)X(t) d t+ε,$ where ψ is a square integrable function defined on [0,1] and ε is an r.r.v. independent of X with zero mean and variance equal to σ².

Hastie and Mallows (1993) introduce an estimator of function ψ based on the minimization of a cubic spline criterion and Marx and Eilers (1996) use a smooth basis of B-splines and then introduce a difference penalty in the log-likelihood in order to derive a P-splines estimator of ψ.

Alternatively, model (1) can be generalized to the case where X is a random variable valued in a real separable Hilbert space H and the relation between X and Y can now be written as $Y=Ψ(X)+ε,$ where Ψ is an element of H′, and H′ is the space of $R$ -valued continuous linear operators defined on H.

Following ideas from Bosq (1991) in the case of ARH processes, we propose in Section 2 below, an estimator of the operator Ψ. This estimator is based on the spectral analysis of the empirical second moment operator of X, which is then inverted in the space spanned by k_n eigenvectors associated with the k_n greatest eigenvalues. The main results are stated in Section 3, that is convergence in probability and almost sure convergence for this estimator. Computational aspects for the method are discussed in Section 4 through a simulation study. A sketch of the proofs are given in Section 5 (detailed proofs may be found in Cardot et al., 1998).

Section snippets

Definition of estimator

The inner product and norm in H are, respectively, denoted by $〈.,.〉_{H}$ and ∥.∥_H and the usual norm ∥.∥_H′ in H′ is defined as $∀T∈H′, ∥T∥_{H′} = sup ∥x∥_{H} =1 |Tx|,$ and satisfies $∀T∈H′, ∥T∥_{H′} = ∑ i∈ N (Te_{i})^{2}^{1/2},$ where $(e_{i})_{i∈N}$ is an orthonormal basis in H. Assuming that the Hilbertian variable X satisfies $E [∥X∥^{2}_{H}]= ∫ Ω ∥X(ω)∥^{2}_{H} d P(ω)<+∞,$ we define (cf. Grenander, 1963), from Riesz's Theorem, the second moment operator Γ of X by $Γ(x)= E (X⊗_{H} X(x))= E (〈X,x〉_{H} X), ∀x∈H.$ The operator Γ is nuclear (and therefore is an Hilbert–Schmidt

Main results

In order to state the main results of the paper, let us introduce the following condition: $(H_{0}) λ ̂_{1} > λ ̂_{2} >⋯> λ ̂_{k_{n}} >0 a.s.,$ which insures almost surely that $Π ̂_{k_{n}} Γ_{n} Π ̂_{k_{n}}$ is regular and its eigenvectors are identifiable. Let us note $(λ_{j})_{j∈N^{∗}}$ the sequence of decreasing eigenvalues of Γ and let us define $a_{j} = 22 λ_{1} −λ_{2} if j=1, a_{j} = 22 min (λ_{j−1} −λ_{j},λ_{j} −λ_{j+1}) if j≠1.$

Theorem 3.1

Suppose that (H₀) and the following hypotheses are satisfied: $(H_{1}) λ_{1} >λ_{2} >⋯>0,$ $(H_{2}) E ∥X∥_{H}^{4} <+∞,$ $(H_{3}) lim n→+∞ nλ_{k_{n}}^{4} =+∞, (H_{3}) lim n→+∞ nλ_{k_{n}}^{2} (∑ j=1 k_{n} a_{j})^{2} =+∞.$ Then $∥ Ψ ̂_{k_{n}} −Ψ∥_{H′} → n→+∞ 0 in$

A simulation study

We have simulated samples $(X_{i},Y_{i}), i=1,…,n$ , from model (1) in which X(t) is a Brownian motion defined on [0,1], ε is normal with mean 0 and variance 0.2 var (Ψ(X)). The Hilbert space H is L²[0,1] and the eigenelements of the covariance operator of X are known to be (see Ash and Gardner, 1975): $λ_{j} = 1 (j−0.5)^{2} π^{2}, V_{j} (t)= 2 sin {(j−0.5) π t}, t∈[0,1], j=1,2,…$ In that case, assumptions (H₃) (respectively (H₆)) on the sequence of eigenvalues in Theorem 3.1 (respectively Theorem 3.2) are fulfilled provided that the

Proof of theorems

Let $(V_{j})_{j∈N^{∗}}$ be a sequence of orthonormal eigenvectors associated with $(λ_{j})_{j∈N^{∗}}$ and let us define in H the operator Ψ_{k_n} as the “theoretical” version of $Ψ ̂_{k_{n}}$ $Ψ_{k_{n}} =ΔΠ_{k_{n}} (Π_{k_{n}} ΓΠ_{k_{n}})^{−1},$ where Π_{k_n} is the orthogonal projection onto the space H_{k_n} spanned by V₁,…,V_{k_n}. First of all, let us remark that $∥Ψ− Ψ ̂_{k_{n}} ∥_{H′} ⩽∥Ψ−Ψ_{k_{n}} ∥_{H′} +∥Ψ_{k_{n}} − Ψ ̂_{k_{n}} ∥_{H′} .$ We have for the first term on the right side of inequality (8) $∥Ψ−Ψ_{k_{n}} ∥_{H′}^{2} = ∑ j=1 ∞ |(Ψ−Ψ_{k_{n}})(V′_{j})|^{2} = ∑ j>k_{n} |Ψ(V′_{j})|^{2},$ where $V′_{j} =(sign 〈 V ̂_{j},V_{j} 〉_{H})V_{j}, j⩾1.$ Since Ψ∈H′, we get $∥Ψ−Ψ_{k_{n}} ∥_{H′} → n→+∞ 0.$

References (14)

J. Dauxois et al.
Asymptotic theory for the principal component analysis of a random vector function: some applications to statistical inference
J. Multivariate Anal.
(1982)
V.V. Yurinskiı̆
Exponential inequalities for sums of random vectors
J. Multivariate Anal.
(1976)
Ash, R.B., Gardner, M.F., 1975. Topics in Stochastic Processes. Academic Press, New...
P. Besse et al.
Approximation spline de la prévision d'un processus fonctionnel autorégressif d'ordre 1
Revue Canadienne de Statistique/Canad. J. Statist.
(1996)
Bosq, D., 1991. Modelization, non-parametric estimation and prediction for continuous time processes. In: Roussas, G....
Cardot, H., 1998. Convergence du lissage spline de la prévision des processus autorégressifs fonctionnels. C.R. Acad....
Cardot, H., Ferraty, F., Sarda, P., 1998. Modèle linéaire fonctionnel. Publ. Lab. Statist. Probab. 04:98, Toulouse,...

There are more references available in the full text version of this article.

Cited by (0)

View full text