2021 | Buch

# Markov Renewal and Piecewise Deterministic Processes

verfasst von: Christiane Cocozza-Thivent

Verlag: Springer International Publishing

Buchreihe: Probability Theory and Stochastic Modelling

2021 | Buch

verfasst von: Christiane Cocozza-Thivent

Verlag: Springer International Publishing

Buchreihe: Probability Theory and Stochastic Modelling

This book is aimed at researchers, graduate students and engineers who would like to be initiated to Piecewise Deterministic Markov Processes (PDMPs). A PDMP models a deterministic mechanism modified by jumps that occur at random times. The fields of applications are numerous : insurance and risk, biology, communication networks, dependability, supply management, etc.

Indeed, the PDMPs studied so far are in fact deterministic functions of CSMPs (Completed Semi-Markov Processes), i.e. semi-Markov processes completed to become Markov processes. This remark leads to considerably broaden the definition of PDMPs and allows their properties to be deduced from those of CSMPs, which are easier to grasp. Stability is studied within a very general framework. In the other chapters, the results become more accurate as the assumptions become more precise. Generalized Chapman-Kolmogorov equations lead to numerical schemes. The last chapter is an opening on processes for which the deterministic flow of the PDMP is replaced with a Markov process.

Marked point processes play a key role throughout this book.

Anzeige

Abstract

Over the few last years, increasing attention has been paid to piecewise deterministic Markov processes (PDMPs) because of their strong potential to describe a variety of situations. They make it possible to represent phenomena with a deterministic behavior that is modified by random jumps.

Abstract

This chapter is an introduction to Markov renewal theory. It contains the fundamental theorem on the change of the observation time and the definition of a completed semi-Markov process (CSMP), on which all future developments will be based.

Abstract

In this chapter, we give more general definitions of PDMPs than the usual ones. First of all, we consider a fully general framework, with only assumptions concerning the deterministic behavior between jumps (namely the flow) and a very natural fundamental link between it and the interarrival distributions, which is the probability distributions of the periods between two consecutive jumps.

Abstract

This chapter provides methods to assess the probability distribution of a hitting time. It is motivated by applications in predictive reliability in which the probability that a piece of equipment is in operation throughout a given period of time has to be evaluated. It is an opportunity to look at new PDMPs that are obtained from the initial model. We give two methods to derive the probability that the hitting time \(\tau \) of B is strictly greater than t.

Abstract

In this chapter we introduce the intensities of marked point processes that will be useful for the future. They give the mean number of times the marks are in a given set during a given time period. They are both tools for proofs and aids for the understanding of formulas, and they provide interesting performance indicators in applications, as we will see in examples.

Abstract

In this chapter we give equations satisfied by the marginal probability distributions of CSMPs, and as a result of PDMPs. They are obtained using the intensities of marked point processes discussed in Chap. 5. As in that chapter, the kernel is progressively specified. The final challenge is to get a formulation that may be used to derive numerical approximation schemes and to prove their convergence, even when there are Dirac measures in the interarrival distributions. This requires Chapman–Kolmogorov equations that are more general than the usual ones, that is, for functions that do not belong to the domain of the extended generator.

Abstract

Contrary to the usual practice, martingales are not at the heart of our study of PDMPs, but a book devoted to PDMPs cannot fail to mention those related to them. This chapter contains the classic results on these martingales. It is based on the results mentioned in Appendix A (Sect. A.4) about the compensator of a marked point process and the associated stochastic calculus. The reader who is not specifically interested in martingales can skip this chapter without prejudicing the reading of the following ones.

Abstract

The purpose of this chapter is to study the relationships between the stability of different processes, especially between the driving Markov chain and the CSMP and consequently between this driving chain and the PDMP. A large proportion of this work is inspired by Jacod [67]. Steady-sate intensities are also derived and applications are given. No special assumption about the structure of the interarrival distributions is required.

Abstract

Monte Carlo simulations are commonly used to get the expectation of quantities related to PDMPs. A rigorous and comprehensive presentation with very interesting examples can be found in De Saporta, Dufour, and Zhang [53].

Abstract

This chapter introduces switching processes (SP). Their construction follows the construction of a PDMP, except that the deterministic flow \(\phi \) is replaced by a stochastic process \(\zeta \), wrongly called “the” intrinsic process. More specifically, for each n, \(\phi (Y_n, \, . \, )\) is replaced by a process whose probability distribution is that of \(\zeta \) given \(\zeta (0)=Y_n\). A PDMP is a switching process.