1. Continuous Time Markov Chain Models for Chemical Reaction Networks

A reaction network is a chemical system involving multiple reactions and chemical species. The simplest stochastic models of such networks treat the system as a continuous time Markov chain with the state being the number of molecules of each species and with reactions modeled as possible transitions of the chain. This chapter is devoted to the mathematical study of such stochastic models. We begin by developing much of the mathematical machinery we need to describe the stochastic models we are most interested in. We show how one can represent counting processes of the type we need in terms of Poisson processes. This random time-change representation gives a stochastic equation for continuous-time Markov chain models. We include a discussion on the relationship between this stochastic equation and the corresponding martingale problem and Kolmogorov forward (master) equation. Next, we exploit the representation of the stochastic equation for chemical reaction networks and, under what we will refer to as the classical scaling, show how to derive the deterministic law of mass action from the Markov chain model. We also review the diffusion, or Langevin, approximation, include a discussion of first order reaction networks, and present a large class of networks, those that are weakly reversible and have a deficiency of zero, that induce product-form stationary distributions. Finally, we discuss models in which the numbers of molecules and/or the reaction rate constants of the system vary over several orders of magnitude. We show that one consequence of this wide variation in scales is that different subsystems may evolve on different time scales and this time-scale variation can be exploited to identify reduced models that capture the behavior of parts of the system. We will discuss systematic ways of identifying the different time scales and deriving the reduced models.