Skip to main content
SpringerLink
Log in

Bayesian Nonparametric Estimation for Reinforced Markov Renewal Processes

  • Published:
Statistical Inference for Stochastic Processes Aims and scope Submit manuscript

Abstract

Starting from the definitions and the properties of reinforced renewal processes and reinforced Markov renewal processes, we characterize, via exchangeability and de Finetti’s representation theorem, a prior that consists of a family of Dirichlet distributions on the space of Markov transition matrices and beta-Stacy processes on distribution functions. Then, we show that this family is conjugate and give some estimate results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

RP:

renewal process

MRP:

Markov renewal process

sMP:

semi-Markov process

RMRP:

reinforced Markov renewal process

RsMP:

reinforced semi-Markov process

References

  • Brock DB (1973) Bayesian analysis for Markov renewal process. Metron 29:185–198

    MATH  Google Scholar 

  • Çinlar E (1975) Markov renewal theory: a survey. Manag Sci J Instit Manage Sci 21:727–752

    Article  MATH  Google Scholar 

  • Duncan GT (1990) Bayes estimation in the Markov renewal process. Bayesian and Likelihood Methods in Statistics and Econometrics: Essays in Honor of George A. Barnard. North-Holland, Amsterdam, pp 265–275

  • Gill RD (1980) Nonparametric estimation based on censored observations of a Markov renewal process. Zeitschrift Wahrscheinlichkeitstheorie Verwandte Gebiete [Became: @J(ProbTher)] 53:97–116

    Article  MATH  MathSciNet  Google Scholar 

  • Gill RD (1994) Lectures on survival analysis. In: Bernard P (eds) Lectures on probability theory (Ecole d’été de Probabilité de Saint-Flour XXII—1992). Springer, Berlin

    Google Scholar 

  • Gill RD, Johansen S (1990) A survey of product-integration with a view toward application in survival analysis. Ann Stat 18:1501–1555

    MATH  MathSciNet  Google Scholar 

  • Greenwood PE, Wefelmeyer W (1996) Empirical estimators for semi-Markov processes. Math Methods Statist 5:299–315

    MATH  MathSciNet  Google Scholar 

  • Hjort NL (1990) Nonparametric bayes estimators based on beta processes in models for life history data. Ann Statis 18(3):1259–1294

    MATH  MathSciNet  Google Scholar 

  • Jain RK (1990) Statistical estimation for semi-Markov models. Metron 48:223–242

    MATH  MathSciNet  Google Scholar 

  • Lagakos SW, Sommer CJ, Zelen M (1978) Semi-Markov models for partially censored data. Biometrika 65:311–318

    Article  MATH  MathSciNet  Google Scholar 

  • Lévy P (1954a) Processus semi-Markoviens. Proc Int Congr Math 3:416–426

    Google Scholar 

  • Lévy P (1954b) Systèmes semi-Markoviens à au plus une infinité dénombrable d’états possibles’. Proc Int Congr Math 2:294

    Google Scholar 

  • Moore EH, Pyke R (1968) Estimation of the transition distributions of a Markov renewal process. Ann Instit Statist, Math 20:411–424

    Article  MATH  MathSciNet  Google Scholar 

  • Muliere P, Secchi P, Walker SG (2000) Urn schemes and reinforced random walks. Stochastic Processes Appl 88(1):59–78

    Article  MATH  MathSciNet  Google Scholar 

  • Muliere P, Secchi P, Walker SG (2003) Reinforced random processes in continuous time. Stochastic Processes Appl 104:117–130

    Article  MATH  MathSciNet  Google Scholar 

  • Phelan MJ (1990a) Bayes estimation from a Markov renewal process. Ann Statist 18:603–616

    MATH  MathSciNet  Google Scholar 

  • Phelan MJ (1990b) Estimating the transition probabilities from censored Markov renewal processes. Statist Probab Lett 10:43–47

    Article  MATH  MathSciNet  Google Scholar 

  • Pyke R (1961a) Markov renewal processes: definitions and preliminary properties. Ann Math Statist 32:1231–1242

    MathSciNet  MATH  Google Scholar 

  • Pyke R (1961b) Markov renewal processes with finitely many states. Ann Math Statist 32:1243–1259

    MathSciNet  MATH  Google Scholar 

  • Pyke R, Schaufele R (1964) Limit theorems for Markov renewal processes. Ann Math Statist 35:1746–1764

    MathSciNet  MATH  Google Scholar 

  • Smith WL (1955) Regenerative stochastic processes. Proc R Soc (Lond) Ser. A 232:6–31

    MATH  Google Scholar 

  • Voelkel JG, Crowley J (1984) Nonparametric inference for a class of semi-Markov processes with censored observations. Ann Statist 12:142–160

    MATH  MathSciNet  Google Scholar 

  • Walker S, Muliere P (1997) Beta-Stacy processes and a generalization of the Pólya-urn scheme. Ann Statist 25:1762–1780

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Università Luigi Bocconi, Milano, Italy

    Paolo Bulla & Pietro Muliere

Authors
  1. Paolo Bulla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pietro Muliere
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paolo Bulla.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bulla, P., Muliere, P. Bayesian Nonparametric Estimation for Reinforced Markov Renewal Processes. Stat Infer Stoch Process 10, 283–303 (2007). https://doi.org/10.1007/s11203-006-9000-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11203-006-9000-x

Keywords

AMS Classification

Search

Navigation