main-content

## Über dieses Buch

The book deals with several closely related topics concerning approxima­ tions and perturbations of random processes and their applications to some important and fascinating classes of problems in the analysis and design of stochastic control systems and nonlinear filters. The basic mathematical methods which are used and developed are those of the theory of weak con­ vergence. The techniques are quite powerful for getting weak convergence or functional limit theorems for broad classes of problems and many of the techniques are new. The original need for some of the techniques which are developed here arose in connection with our study of the particular applica­ tions in this book, and related problems of approximation in control theory, but it will be clear that they have numerous applications elsewhere in weak convergence and process approximation theory. The book is a continuation of the author's long term interest in problems of the approximation of stochastic processes and its applications to problems arising in control and communication theory and related areas. In fact, the techniques used here can be fruitfully applied to many other areas. The basic random processes of interest can be described by solutions to either (multiple time scale) Ito differential equations driven by wide band or state dependent wide band noise or which are singularly perturbed. They might be controlled or not, and their state values might be fully observable or not (e. g. , as in the nonlinear filtering problem).

## Inhaltsverzeichnis

### 1. Weak Convergence

Abstract
The basic questions dealt with in this book concern approximations of relatively complex processes by simpler and more tractable processes. These processes might be controlled or not controlled. The approximation might be of interest for either the purposes of engineering design, or for other analytical or numerical work. The theory of weak convergence of measures seems to be the fundamental and most widely used and successful tool for such purposes. See, for example, the techniques, examples and references in the books [B6, E2, K20]. In this chapter, we will discuss the basic ideas in the subject which will be useful for our work in the sequel. The detailed examples will not be dealt with in this chapter, but are left to the rest of the book. The basic techniques discussed in this chapter will all be further developed and worked out in detail in connection with the applications dealt with in the succeeding chapters. The concepts and criteria have a rather abstract flavor as they are initially introduced, but when applied in the succeeding chapters, they take on concrete and readily usable forms.
Harold J. Kushner

### 2. Stochastic Processes: Background

Abstract
In this chapter, we survey some of the concepts in the theory of stochastic processes which will be used in the sequel. The material is intended to be only a discussion of some of the main ideas which will be needed and to serve as a convenient reference. It is not intended to be complete in any sense. Additional details, motivation and background can be found in the references. The reader familiar with the basic facts concerning stochastic differential equations can skip the chapter and refer to the results as needed. Section 1 gives some definitions and facts concerning martingales. In Section 2, we define stochastic integrals with respect to a Wiener process and state Itô’s Formula, the fundamental tool in the stochastic calculus. Section 3 uses the stochastic integrals to define stochastic differential equations (SDE) of the diffusion type, and obtains bounds on their solutions which will be used later in the study of properties of the solution processes, as well as in the proofs of tightness of sequences of solutions to such equations. In Section 4, we discuss three standard methods for obtaining existence and uniqueness of solutions to SDE’s. These sections give some flavor of a few of the basic ideas. But they just touch the surface of the subject. Sections 5 and 6 concern an alternative and very useful method for verifying whether a process satisfies an SDE of the diffusion type. The so-called “martingale problem” method which is discussed in these sections is perhaps the most useful approach to showing that the limit of a weakly convergent sequence of processes is, indeed, a solution to a SDE.
Harold J. Kushner

### 3. Controlled Stochastic Differential Equations

Abstract
In this chapter, we introduce the stochastic control problem and discuss various classes of controls, approximations to these classes and questions concerning the existence of an optimal control. We work with one particular type of cost functional for simplicity in the development. In Section 1, ordinary admissible controls are introduced and it is shown why they might not be adequate for our needs. Section 2 introduces the notion of “relaxed control” for deterministic problems and shows how to use them to prove the existence of an optimal control. It is also shown that these “generalized” or relaxed controls can be approximated by piecewise constant ordinary controls.
Harold J. Kushner

### 4. Controlled Singularly Perturbed Systems

Abstract
Many control problems can be modelled by systems of differential equations where the state variable can be divided into two coupled groups. Those in the first group change at a “normal” rate, and those in the second group change at a much “faster” rate. Such systems can be loosely termed “two time scale” systems, and have been the subject of a great deal of work. We refer the reader to [B2, K4, K5], and the references contained therein. For linear systems models, the first group might represent the low frequency part of the system and the second group the high frequency part.
Harold J. Kushner

### 5. Functional Occupation Measures and Average Cost per Unit Time Problems

Abstract
In Chapter 4.4, we dealt with average cost per unit time problems for singularly perturbed systems. A basic question concerned the characterization of the limit (as ϵ → 0, T → ∞) pathwise cost as the cost associated with an average cost per unit time problem for the averaged system. The concept of feedback relaxed control was introduced and it was shown, under given conditions, that the limit costs were the average costs per unit time for some averaged system with such a feedback relaxed control. The feedback relaxed control was introduced for mathematical reasons, since it allows a characterization of the limit, which then allowed us to prove the “approximate optimality” theorems.
Harold J. Kushner

### 6. The Nonlinear Filtering Problem

Abstract
In Chapter 4, we were concerned with the problem of approximations for the singularly perturbed system (4.1.1), (4.1.2), and with the associated problem of control approximations. In this chapter, we will be concerned with approximations for the nonlinear filtering problem. Suppose that we observe the noise corrupted data yϵ(•) defined by $$d{y^\varepsilon } = g({x^\varepsilon },{z^\varepsilon })dt + d{w_0}$$, where w0(•) is a standard vector-valued Wiener process. Owing to the complexity and high dimension of the original system (4.1.1), (4.1.2), the construction of the optimal filter or even the direct construction of an acceptable approximation can be a very hard task. The possibility of using some sort of averaging method, such as in Chapter 4, to get a simpler filter (say, one for an averaged system) is quite appealing. One would use the filter for the averaged system, but the input would be the true physical observations yϵ(•).
Harold J. Kushner

### 7. Weak Convergence: The Perturbed Test Function Method

Abstract
The methods which were used in Chapters 3–6 to characterize the limits of $$\left\{ {{x^\varepsilon }\left( \cdot \right),{m^\varepsilon }\left( \cdot \right)} \right\}$$ as solutions to the averaged system are very well suited to the types of “white noise driven” Itô equations which were used there. For the singular perturbation problem of Chapter 4, with model (4.1.1), (4.1.2) with control mϵ(·), the martingale method which was used to characterize the limit process can be roughly described as follows (see Theorem 4.1.2): We apply the differential operator Aϵ of the (xϵ(·), zϵ(·)) process to a nice test function f (·). Then for continuous and bounded functions h(·) and ø j (·), and for positive times t i tt + s, Itô’s formula was used to show that for each ϵ
$$Eh\left( {{x^{\varepsilon }}\left( {{t_{i}}} \right),{{\left( {{\phi _{j}},{m^{\varepsilon }}} \right)}_{{{t_{i}}}}},i \leqslant q,j \leqslant p} \right)\left[ {f\left( {{x^{\varepsilon }}\left( {t + s} \right)} \right) - f\left( {{x^{\varepsilon }}\left( t \right)} \right) - \int_{t}^{{t + s}} {\left( {{A^{\varepsilon }}f} \right)\left( {{x^{\varepsilon }}\left( u \right),{z^{\varepsilon }}\left( u \right),u} \right)du} } \right] = 0$$
(0.1)
.
Harold J. Kushner

### 8. Singularly Perturbed Wide-Band Noise Driven Systems

Abstract
In this chapter, the ideas of Chapter 7 will be used to extend the results in Chapters 4 and 5 to the singular perturbation problem for wide band noise driven systems. The actual physical models for many practical systems are driven by wide band noise and it is important to know how to approximate these models and the associated control problems by simple diffusion models and control problems.
Harold J. Kushner

### 9. Stability Theory

Abstract
The tightness assumptions (A4.1.6) or (A4.4.2) on {zϵ(t), t < ∞, ϵ > 0} or of (A4.4.1) on {xϵ(t),t < ∞, ϵ > 0} are essentially questions of stochastic stability. Of course, if the state spaces are bounded, then the cited as­sumptions are automatically satisfied. The deterministic specialization of the above cited tightness requirements is that the trajectories of interest (those of xϵ(ּ) and/or xϵ(ּ)) be bounded on the time interval of interest. To prove that boundedness in particular cases for the deterministic problem, some form of Liapunov function method is usually used. Stochastic “Liapunov function methods” are also very useful (if not indispensible at this time) to prove the required tightness for the stochastic problems, and we will discuss several approaches in this chapter.
Harold J. Kushner

### 10. Parametric Singularities

Abstract
Multiple time scale systems can arise in applications due to the effects of small parameters, and this gives rise to a class of systems which are quite different from the models (4.1.1), (4.1.2), or their wide band noise driven analogs. Consider the following case:
$$in {y^{\varepsilon }} + {a_{2}}\dot{y} + {a_{1}}{y^{\varepsilon }} = u(t) + whitenoise$$
(0.1)
where a i > 0, ϵ > 0 and u(t) is a control. The small parameter ϵ might represent, for example, a small inductance in an electric circuit. Define x ϵ 1 = y ϵ and xϵ 2 = y ϵ . Then, we can write
$$\begin{gathered} dx_1^\varepsilon = x_2^\varepsilon dt \hfill \\ \varepsilon dx_2^\varepsilon = [ - {a_2}x_2^\varepsilon - {a_1}x_1^\varepsilon + u\left( t \right)]dt + {\sigma _2}dw \hfill \\ \end{gathered}$$
(0.2)
for some constant σ2 > 0. It will be seen in Section 1 that x ϵ 2 () is a “wide band noise” process which converges to white noise in the sense that its integral converges to a Wiener process with a state and control dependent bias.
Harold J. Kushner

### Backmatter

Weitere Informationen