Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top

2015 | Book

Introduction Read chapter Read first chapter

Linear Stochastic Systems

A Geometric Approach to Modeling, Estimation and Identification

Authors: Anders Lindquist, Giorgio Picci

Publisher: Springer Berlin Heidelberg

Book Series : Series in Contemporary Mathematics

Part of: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Login to get access
insite
SEARCH

About this book

This book presents a treatise on the theory and modeling of second-order stationary processes, including an exposition on selected application areas that are important in the engineering and applied sciences. The foundational issues regarding stationary processes dealt with in the beginning of the book have a long history, starting in the 1940s with the work of Kolmogorov, Wiener, Cramér and his students, in particular Wold, and have since been refined and complemented by many others. Problems concerning the filtering and modeling of stationary random signals and systems have also been addressed and studied, fostered by the advent of modern digital computers, since the fundamental work of R.E. Kalman in the early 1960s. The book offers a unified and logically consistent view of the subject based on simple ideas from Hilbert space geometry and coordinate-free thinking. In this framework, the concepts of stochastic state space and state space modeling, based on the notion of the conditional independence of past and future flows of the relevant signals, are revealed to be fundamentally unifying ideas. The book, based on over 30 years of original research, represents a valuable contribution that will inform the fields of stochastic modeling, estimation, system identification, and time series analysis for decades to come. It also provides the mathematical tools needed to grasp and analyze the structures of algorithms in stochastic systems theory.

Table of Contents

Frontmatter
Chapter 1. Introduction
Abstract
In this book we consider the following inverse problem: Given a stationary stochastic vector process, find a linear stochastic system, driven by white noise, having the given process as its output. This stochastic realization problem is a problem of state-space modeling, and like most other inverse problems it has in general infinitely many solutions. Parametrizing these solutions and describing them in a systems-theoretic context is an important problem from the point of view of applications.
Anders Lindquist, Giorgio Picci
Chapter 2. Geometry of Second-Order Random Processes
Abstract
In this book, modeling and estimation problems of random processes are treated in a unified geometric framework. For this, we need some basic facts about the Hilbert space theory of stochastic vector processes that have finite second order moments and are stationary in the wide sense. Such a process \(\{y(t)\}_{t\in \mathbb{Z}}\) is a collection of real or complex-valued random variables \(y_{k}(t),k = 1,2,\ldots,m,t \in \mathbb{Z}\), which generate a Hilbert space H with inner product
$$\displaystyle{\langle \xi,\eta \rangle =\mathop{ \mathrm{E}}\nolimits \{\xi \bar{\eta }\},}$$
where bar denotes conjugation. This Hilbert space is endowed with a shift, i.e., a unitary operator \(\mathcal{U}:\, \mathbf{H} \rightarrow \mathbf{H}\) with the property that
$$\displaystyle{y_{k}(t + 1) = \mathcal{U}y_{k}(t),\quad k = 1,2,\ldots,t \in \mathbb{Z}.}$$
In this chapter we introduce some basic geometric facts for such Hilbert spaces. Although we shall assume that the reader has some knowledge of elementary Hilbert space theory, for the benefit of the reader, some relevant facts are collected in Appendix B.2.
Anders Lindquist, Giorgio Picci
Chapter 3. Spectral Representation of Stationary Processes
Abstract
In this chapter we review the spectral representation of stationary processes. This representation theory is useful for at least two reasons. First it leads to concrete representation results of stationary processes in terms of white noise. These representations are basic for filtering and prediction and also for state-space modeling of random signals. Second, spectral representation theory provides a functional calculus for random variables and processes in terms of functions of a complex variable, much in the same spirit as the Fourier transform for deterministic signals. Unfortunately the Fourier transform of a stationary process cannot be defined in a deterministic pathwise sense. For it is well-known that the sample paths of a discrete-time stationary Gaussian process of, say, independent random variables (discrete time white noise) are neither in ℓ 2 nor uniformly bounded with probability one, and hence as functions of time they do not admit a Fourier transform [129].
Anders Lindquist, Giorgio Picci
Chapter 4. Innovations, Wold Decomposition, and Spectral Factorization
Abstract
We begin this chapter by reviewing some basic concepts from the theory of dynamic estimation in the classical setup of Wiener and Kolmogorov. The theory leads naturally to considering certain white noise representations of the observation process, which are prototypes of stochastic dynamical systems described in input-output form. These representations were first introduced in geometric terms in the seminal work of H. Wold on stationary processes and prediction theory. Wold’s ideas have been generalized in many directions. One such generalization will be discussed in this chapter and will form the basis of representation theorems which will be used throughout the book. Generalizations of Wold decomposition have become part of functional analysis and have led to a unifying view of certain fundamental problems in operator theory and Hardy spaces. The operator theoretic (and Hardy space) results which stem from this idea can, in a sense, be seen as function-analytic counterparts of results in the theory of stationary processes and in prediction theory. In Sect. 4.6 we take advantage of this conceptual connection to review, in an economical and essentially self-contained way, some basic parts of Hardy space theory that will be needed in various parts of the book.
Anders Lindquist, Giorgio Picci
Chapter 5. Spectral Factorization in Continuous Time
Abstract
In this chapter we shall describe the continuous-time analogs of the ideas and representation results of the previous chapter. As mentioned in Sect. 2.​8, the interesting generalization of the discrete-time setting is to continuous-time stationary increments processes. For this reason we shall be mostly concerned with this class.
Anders Lindquist, Giorgio Picci
Chapter 6. Linear Finite-Dimensional Stochastic Systems
Abstract
This chapter is an introduction to linear state-space modeling of second-order, wide-sense stationary, stochastic vector processes. In particular, we shall discuss modeling of discrete-time purely non deterministic processes with a rational spectral density matrix. These processes turn out to admit representations as the output y of a finite-dimensional linear system
$$\displaystyle{ \left \{\begin{array}{lcl} x(t + 1) &=& Ax(t) + Bw(t)\quad \\ y(t) &=& Cx(t) + Dw(t)\quad \end{array} \right. }$$
driven by a white noise input {w(t)}, where A, B, C and D are constant matrices of appropriate dimensions. These state-space descriptions provide a natural and useful class of parametrized stochastic models widely used in control and signal processing, leading to simple recursive estimation algorithms. Stochastic realization theory consists in characterizing and determining any such representation. This is in turn related to spectral factorization. The structure of these stochastic models is described in geometric terms based on coordinate-free representations and on elementary Hilbert space concepts.
Anders Lindquist, Giorgio Picci
Chapter 7. The Geometry of Splitting Subspaces
Abstract
The purpose of this chapter is to introduce coordinate-free representations of a stationary process y by constructing state spaces from basic principles. This will in particular accommodate both finite- and infinite-dimensional stochastic systems.
Anders Lindquist, Giorgio Picci
Chapter 8. Markovian Representations
Abstract
The purpose of this chapter is to introduce coordinate-free representations of a stationary process y by constructing state spaces from basic principles. This will in particular accommodate both finite- and infinite-dimensional stochastic systems.
Anders Lindquist, Giorgio Picci
Chapter 9. Proper Markovian Representations in Hardy Space
Abstract
In this chapter we reformulate the splitting geometry in terms of functional models in Hardy space. This allows us to use the power of Hardy space theory to prove several additional results and useful characterizations. We shall only deal with proper Markovian representation, and therefore we formulate several functional criteria for properness.
Anders Lindquist, Giorgio Picci
Chapter 10. Stochastic Realization Theory in Continuous Time
Abstract
This chapter is devoted to continuous-time versions of the basic results in Chaps. 6, 8 and 9 In this context, the linear stochastic model (6.1) corresponds to a system
$$\displaystyle{ \left \{\begin{array}{@{}l@{\quad }l@{}} dx = Axdt + Bdw\quad \\ dy = Cxdt + Ddw\quad \end{array} \right. }$$
of stochastic differential equations driven by the increments of a vector Wiener process w. The state process x will still be a stationary process, but the output process y has stationary increments. In the case when D = 0, we may instead consider a model
$$\displaystyle{ \left \{\begin{array}{@{}l@{\quad }l@{}} dx = Axdt + Bdw\quad \\ \phantom{d}y = Cx\quad \end{array} \right. }$$
for which the output is a stationary process.
Anders Lindquist, Giorgio Picci
Chapter 11. Stochastic Balancing and Model Reduction
Abstract
Suppose that we are given a stochastic model of the form
$$\displaystyle{ \left \{\begin{array}{lcl} x(t + 1) &=& \mathit{Ax}(t) + \mathit{Bw}(t)\quad \\ y(t) &=& \mathit{Cx } (t) + \mathit{Dw } (t)\quad \end{array} \right.\!\!\!\!\!\!\!, }$$
(11.1)
perhaps coming from the description of a certain physical or engineering problem. Without much loss of generality, we shall assume that (11.1) is a minimal stochastic realization, and consequently that \(\mathbf{X} =\mathop{ \mathrm{span\,}}\nolimits \{x_{k}(0),\,k = 1,\ldots,n\}\) is a minimal Markovian splitting subspace.
Anders Lindquist, Giorgio Picci
Chapter 12. Finite-Interval and Partial Stochastic Realization Theory
Abstract
In Sect. 11.2 we described a procedure for computing the triplet .A;C;CN / of a stationary process y starting from the Hankel matrix H1 formed from the infinite covariance sequence .ƒ0;ƒ1;ƒ2; : : : /. This procedure leads, via the solution of the associated linear matrix inequality (6.102), to the construction of all minimal stochastic realizations of the process.
Anders Lindquist, Giorgio Picci
Chapter 13. Subspace Identification of Time Series
Abstract
In this chapter we consider the identification problem of constructing dynamical models of observed signals starting from a sequence of experimental data (called a “time-series” in statistics). More precisely, given a finite observed sample
$$\displaystyle{ (y_{0},y_{1},y_{2},\ldots,y_{N}) }$$
(13.1)
we want to estimate the parameters (A, B, C, D) of a linear stochastic system
$$\displaystyle{ \left \{\begin{array}{@{}l@{\quad }l@{}} x(t + 1) = \mathit{Ax}(t) + \mathit{Bw}(t) \quad \\ y(t)\phantom{ + 12} = \mathit{Cx } (t) + \mathit{Dw } (t)\quad \end{array} \right. }$$
(13.2)
which explains the data, in the sense that the sequence (13.1) is modeled as a sample trajectory of the output process y. Since it is impossible to distinguish individual minimal Markovian representations in the class \(\mathcal{M}\) (defined in Chap. 8) from output data, the best we can do is to determine a representative from this class, for which we choose the forward realization of the predictor space \(\mathbf{X}_{-} =\mathop{ \mathrm{E}}\nolimits ^{\mathbf{H}^{-} }\mathbf{H}^{+}\).
Anders Lindquist, Giorgio Picci
Chapter 14. Zero Dynamics and the Geometry of the Riccati Inequality
Abstract
In this chapter we consider minimal, finite-dimensional stochastic systems both in discrete and continuous time. We show how the zero structure of the minimal spectral factors relate to the splitting subspace geometry of stationary stochastic models and to the corresponding algebraic Riccati inequality. We introduce the notion of output-induced subspace of a minimal Markovian splitting subspace, which is the stochastic analogue of the output-nulling subspace in geometric control theory [23, 316]. Through this concept the analysis can be made coordinate-free, and straightforward geometric methods can be applied.
Anders Lindquist, Giorgio Picci
Chapter 15. Smoothing and Interpolation
Abstract
Given a linear stochastic system of dimension n in either discrete or continuous time, the smoothing problem amounts to determining the least-squares estimates
$$\displaystyle{\hat{x}(t) =\mathop{ \mathrm{E}}\nolimits \{x(t)\mid y(s);\;t_{0} \leq s \leq t_{1}\},\quad t_{0} \leq t \leq t_{1}}$$
for some finite interval [t 0, t 1]. When t 0 → −∞ and t 1 → ∞, we end up in the stationary setting of Sect. 14.​3, and we shall use this fact to reduce the dimension of the smoothing algorithms.
Anders Lindquist, Giorgio Picci
Chapter 16. Acausal Linear Stochastic Models and Spectral Factorization
Abstract
Most of the results in this book deal with models of stationary processes in terms of a forward and a backward stochastic realization with a few deviations to show that similar results hold also in a nonstationary setting. The purpose of this chapter is to show that, still remaining within the circle of ideas of stochastic modeling, practically all results on stochastic realization presented in the literature do in fact generalize to cover quite arbitrary causality structures.
Anders Lindquist, Giorgio Picci
Chapter 17. Stochastic Systems with Inputs
Abstract
In this chapter we study the stochastic realization problem with inputs. Our aim is to provide procedures for constructing state space models for a stationary process y driven by an exogenous observable input signal u, also modeled as a stationary process.
Anders Lindquist, Giorgio Picci
Backmatter
Metadata
Title
Linear Stochastic Systems
Authors
Anders Lindquist
Giorgio Picci
Copyright Year
2015
Publisher
Springer Berlin Heidelberg
Electronic ISBN
978-3-662-45750-4
Print ISBN
978-3-662-45749-8
DOI
https://doi.org/10.1007/978-3-662-45750-4