main-content

## Über dieses Buch

This book was written for an introductory one-semester or two-quarter course in stochastic processes and their applications. The reader is assumed to have a basic knowledge of analysis and linear algebra at an undergraduate level. Stochastic models are applied in many fields such as engineering systems, physics, biology, operations research, business, economics, psychology, and linguistics. Stochastic modeling is one of the promising kinds of modeling in applied probability theory. This book is intended to introduce basic stochastic processes: Poisson pro­ cesses, renewal processes, discrete-time Markov chains, continuous-time Markov chains, and Markov-renewal processes. These basic processes are introduced from the viewpoint of elementary mathematics without going into rigorous treatments. This book also introduces applied stochastic system modeling such as reliability and queueing modeling. Chapters 1 and 2 deal with probability theory, which is basic and prerequisite to the following chapters. Many important concepts of probabilities, random variables, and probability distributions are introduced. Chapter 3 develops the Poisson process, which is one of the basic and im­ portant stochastic processes. Chapter 4 presents the renewal process. Renewal­ theoretic arguments are then used to analyze applied stochastic models. Chapter 5 develops discrete-time Markov chains. Following Chapter 5, Chapter 6 deals with continuous-time Markov chains. Continuous-time Markov chains have im­ portant applications to queueing models as seen in Chapter 9. A one-semester course or two-quarter course consists of a brief review of Chapters 1 and 2, fol­ lowed in order by Chapters 3 through 6.

## Inhaltsverzeichnis

### Chapter 1. Probability Theory

Abstract
We cannot predict in advance the outcome of tossing a coin or a die, drawing a card, and so on. Such an experiment is called a random trial since nobody knows the outcome in advance and it seems to be “random”. However, we can know all the possible outcomes of a random trial in advance. For instance, if we consider tossing a coin, the possible outcomes are “H(heads)” and “T(tails)”. Knowing all the possible outcomes in advance, we should assign each outcome with a number in advance; this is its probability.
Shunji Osaki

### Chapter 2. Random Variables and Distributions

Abstract
Consider the random phenomenon of a queue at an Automated Teller’s Machine (ATM) at a bank, where a queue is a waiting line. Potential customers arrive at the ATM to get cash or other services. Table 2.1.1 shows the observations of the moments of arrival of customers over the course of an hour. Two figures can be considered to describe the random phenomena in Table 2.1.1.
Shunji Osaki

### Chapter 3. Poisson Processes

Abstract
As shown in Section 2.1, a stochastic process can be described by the laws of probability at each point of time t ≥ 0. As shown in Fig. 2.1.1, we are very much interested in the random variable N(t), which denotes the number of arriving customers up to time t,where N(t) = 0, 1, 2,…. A counting process {N(t), t≥ 0} is one of the stochastic processes, and Fig. 2.1.1 shows a “sample function” or “sample path” of the counting process {N(t), t ≥ 0}. We can consider several examples of counting processes, where the “customer” is replaced by other relevant words such as the “call” in congestion theory, the “failure” of machines, and the arriving “job” or arriving “transaction” of computer systems.
Shunji Osaki

### Chapter 4. Renewal Processes

Abstract
In the preceding chapter we have introduced the Poisson process from two different viewpoints. According to Definition 3.2.2 the Poisson process has stationary independent increments and the probability that an event takes place for a small interval h is λh, where the proportional constant λ is the parameter of the process. Furthermore, according to Definition 3.3.2 the Poisson process is a renewal process in which the interarrival times are independent and identically distributed exponentially with the mean 1/λ. Figure 4.1.1 shows two realizations of the Poisson process from these two viewpoints.
Shunji Osaki

### Chapter 5. Discrete-Time Markov Chains

Abstract
In the preceding two chapters we have discussed two continuous-time stochastic processes, i.e., the Poisson process and the renewal process. In this and in following chapters we shall discuss Markov chains. Recall that we are considering stochastic processes with discrete-state space throughout this book.
Shunji Osaki

### Chapter 6. Continuous-Time Markov Chains

Abstract
In this chapter we again develop continuous-time stochastic processes having Markov properties.
Shunji Osaki

### Chapter 7. Markov Renewal Processes

Abstract
We have discussed Markov chains in the preceding two chapters. In Chapter 5 we discussed discrete-time Markov chains in which the process can move from one state to another (including to itself) in discrete time. In Chapter 6 we discussed continuous-time Markov chains in which the process can move from one state to another, where each interarrival time is distributed exponentially. Note that only the exponential distribution has the memoryless property which plays an important role in analyzing the process.
Shunji Osaki

### Chapter 8. Reliability Models

Abstract
In the preceding chapters we have introduced several stochastic processes and developed their properties. In this chapter we discuss reliability models by using the results of the preceding chapters. There are fruitful applications of stochastic processes in reliability models.
Shunji Osaki

### Chapter 9. Queueing Models

Abstract
We observe that people are waiting for service. Typical examples of such waiting lines are found at a supermarket, a restaurant, a bank, and so on. Such waiting lines are real waiting lines which we can observe directly. However, there are several waiting lines which we cannot observe directly. For instance, we often fail to complete a telephone call because the line is busy. An aircraft has to wait for its landing order because the runway is busy. A job or transaction has to wait its processing turn because the processor is busy.
Shunji Osaki

### Backmatter

Weitere Informationen