Skip to main content
Top
Published in: Lifetime Data Analysis 1/2024

17-11-2023

Assessing model prediction performance for the expected cumulative number of recurrent events

Author: Olivier Bouaziz

Published in: Lifetime Data Analysis | Issue 1/2024

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In a recurrent event setting, we introduce a new score designed to evaluate the prediction ability, for a given model, of the expected cumulative number of recurrent events. This score can be seen as an extension of the Brier Score for single time to event data but works for recurrent events with or without a terminal event. Theoretical results are provided that show that under standard assumptions in a recurrent event context, our score can be asymptotically decomposed as the sum of the theoretical mean squared error between the model and the true expected cumulative number of recurrent events and an inseparability term that does not depend on the model. This decomposition is further illustrated on simulations studies. It is also shown that this score should be used in comparison with a reference model, such as a nonparametric estimator that does not include the covariates. Finally, the score is applied for the prediction of hospitalisations on a dataset of patients suffering from atrial fibrillation and a comparison of the prediction performances of different models, such as the Cox model, the Aalen Model or the Ghosh and Lin model, is investigated.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
go back to reference Andersen PK, Angst J, Ravn H (2019) Modeling marginal features in studies of recurrent events in the presence of a terminal event. Lifetime Data Anal 25:681–695 MathSciNetCrossRef Andersen PK, Angst J, Ravn H (2019) Modeling marginal features in studies of recurrent events in the presence of a terminal event. Lifetime Data Anal 25:681–695 MathSciNetCrossRef
go back to reference Andersen PK, Borgan Ø, Gill RD, Keiding N (1993) Statistical models based on counting processes. Springer series in statistics. Springer-Verlag, New York CrossRef Andersen PK, Borgan Ø, Gill RD, Keiding N (1993) Statistical models based on counting processes. Springer series in statistics. Springer-Verlag, New York CrossRef
go back to reference Andersen PK, Gill RD (1982) Cox’s regression model for counting processes: a large sample study. Ann Stat 10(4):1100–1120 MathSciNetCrossRef Andersen PK, Gill RD (1982) Cox’s regression model for counting processes: a large sample study. Ann Stat 10(4):1100–1120 MathSciNetCrossRef
go back to reference Bouaziz O, Geffray S, Lopez O (2015) Semiparametric inference for the recurrent events process by means of a single-index model. Statistics 49:361–385 MathSciNetCrossRef Bouaziz O, Geffray S, Lopez O (2015) Semiparametric inference for the recurrent events process by means of a single-index model. Statistics 49:361–385 MathSciNetCrossRef
go back to reference Bouaziz O, Lopez O (2010) Conditional density estimation in a censored single-index regression model. Bernoulli 16:514–542 MathSciNetCrossRef Bouaziz O, Lopez O (2010) Conditional density estimation in a censored single-index regression model. Bernoulli 16:514–542 MathSciNetCrossRef
go back to reference Bradley AA, Schwartz SS, Hashino T (2008) Sampling uncertainty and confidence intervals for the brier score and brier skill score. Weather Forecast 23:992–1006 CrossRef Bradley AA, Schwartz SS, Hashino T (2008) Sampling uncertainty and confidence intervals for the brier score and brier skill score. Weather Forecast 23:992–1006 CrossRef
go back to reference Cook RJ, Lawless J (2007) The statistical analysis of recurrent events. Springer Science & Business Media, New-York, USA Cook RJ, Lawless J (2007) The statistical analysis of recurrent events. Springer Science & Business Media, New-York, USA
go back to reference Cook RJ, Lawless JF (1997) Marginal analysis of recurrent events and a terminating event. Stat Med 16:911–924 CrossRef Cook RJ, Lawless JF (1997) Marginal analysis of recurrent events and a terminating event. Stat Med 16:911–924 CrossRef
go back to reference Cox DR (1972) Regression models and life-tables. J R Stat Soc Ser B (Methodol) 34:187–202 MathSciNet Cox DR (1972) Regression models and life-tables. J R Stat Soc Ser B (Methodol) 34:187–202 MathSciNet
go back to reference Dabrowska DM (1989) Uniform consistency of the kernel conditional Kaplan-Meier estimate. Ann. Stat 17(3):1157–1167 MathSciNetCrossRef Dabrowska DM (1989) Uniform consistency of the kernel conditional Kaplan-Meier estimate. Ann. Stat 17(3):1157–1167 MathSciNetCrossRef
go back to reference Gerds TA, Kattan MW, Schumacher M, Yu C (2013) Estimating a time-dependent concordance index for survival prediction models with covariate dependent censoring. Stat Med 32:2173–2184 MathSciNetCrossRef Gerds TA, Kattan MW, Schumacher M, Yu C (2013) Estimating a time-dependent concordance index for survival prediction models with covariate dependent censoring. Stat Med 32:2173–2184 MathSciNetCrossRef
go back to reference Gerds TA, Schumacher M (2006) Consistent estimation of the expected brier score in general survival models with right-censored event times. Biom J 48:1029–1040 MathSciNetCrossRef Gerds TA, Schumacher M (2006) Consistent estimation of the expected brier score in general survival models with right-censored event times. Biom J 48:1029–1040 MathSciNetCrossRef
go back to reference Ghosh D, Lin D (2003) Semiparametric analysis of recurrent events data in the presence of dependent censoring. Biometrics 59:877–885 MathSciNetCrossRef Ghosh D, Lin D (2003) Semiparametric analysis of recurrent events data in the presence of dependent censoring. Biometrics 59:877–885 MathSciNetCrossRef
go back to reference Ghosh D, Lin DY (2002) Marginal regression models for recurrent and terminal events. Stat Sin 12(3):663–688 MathSciNet Ghosh D, Lin DY (2002) Marginal regression models for recurrent and terminal events. Stat Sin 12(3):663–688 MathSciNet
go back to reference Graf E, Schmoor C, Sauerbrei W, Schumacher M (1999) Assessment and comparison of prognostic classification schemes for survival data. Stat Med 18:2529–2545 CrossRef Graf E, Schmoor C, Sauerbrei W, Schumacher M (1999) Assessment and comparison of prognostic classification schemes for survival data. Stat Med 18:2529–2545 CrossRef
go back to reference Harrell FE Jr, Lee KL, Mark DB (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:361–387 CrossRef Harrell FE Jr, Lee KL, Mark DB (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:361–387 CrossRef
go back to reference Hougaard P, Hougaard P (2000) Analysis of multivariate survival data, vol 564. Springer, New York, USA CrossRef Hougaard P, Hougaard P (2000) Analysis of multivariate survival data, vol 564. Springer, New York, USA CrossRef
go back to reference Kalbfleisch JD, Prentice RL (2002) The statistical analysis of failure time data, 2nd edn. Wiley series in probability and statistics. Wiley-Interscience (John Wiley & Sons), Hoboken CrossRef Kalbfleisch JD, Prentice RL (2002) The statistical analysis of failure time data, 2nd edn. Wiley series in probability and statistics. Wiley-Interscience (John Wiley & Sons), Hoboken CrossRef
go back to reference Lin DY, Wei L-J, Yang I, Ying Z (2000) Semiparametric regression for the mean and rate functions of recurrent events. J R Stat Soc Ser B (Stat Methodol) 62:711–730 MathSciNetCrossRef Lin DY, Wei L-J, Yang I, Ying Z (2000) Semiparametric regression for the mean and rate functions of recurrent events. J R Stat Soc Ser B (Stat Methodol) 62:711–730 MathSciNetCrossRef
go back to reference Liu L, Wolfe RA, Huang X (2004) Shared frailty models for recurrent events and a terminal event. Biometrics 60:747–756 MathSciNetCrossRef Liu L, Wolfe RA, Huang X (2004) Shared frailty models for recurrent events and a terminal event. Biometrics 60:747–756 MathSciNetCrossRef
go back to reference Prentice RL, Williams BJ, Peterson AV (1981) On the regression analysis of multivariate failure time data. Biometrika 68:373–379 MathSciNetCrossRef Prentice RL, Williams BJ, Peterson AV (1981) On the regression analysis of multivariate failure time data. Biometrika 68:373–379 MathSciNetCrossRef
go back to reference Rondeau V, Mathoulin-Pelissier S, Jacqmin-Gadda H, Brouste V, Soubeyran P (2007) Joint frailty models for recurring events and death using maximum penalized likelihood estimation: application on cancer events. Biostatistics 8:708–721 CrossRef Rondeau V, Mathoulin-Pelissier S, Jacqmin-Gadda H, Brouste V, Soubeyran P (2007) Joint frailty models for recurring events and death using maximum penalized likelihood estimation: application on cancer events. Biostatistics 8:708–721 CrossRef
go back to reference Scheike TH (2002) The additive nonparametric and semiparametric Aalen model as the rate function for a counting process. Lifetime Data Anal 8:247–262 MathSciNetCrossRef Scheike TH (2002) The additive nonparametric and semiparametric Aalen model as the rate function for a counting process. Lifetime Data Anal 8:247–262 MathSciNetCrossRef
go back to reference Schoop R, Schumacher M, Graf E (2011) Measures of prediction error for survival data with longitudinal covariates. Biom J 53:275–293 MathSciNetCrossRef Schoop R, Schumacher M, Graf E (2011) Measures of prediction error for survival data with longitudinal covariates. Biom J 53:275–293 MathSciNetCrossRef
go back to reference Schroder J, Bouaziz O, Agner BR, Martinussen T, Madsen PL, Li D, Dixen U (2019) Recurrent event survival analysis predicts future risk of hospitalization in patients with paroxysmal and persistent atrial fibrillation. PLoS One 14:e0217983 CrossRef Schroder J, Bouaziz O, Agner BR, Martinussen T, Madsen PL, Li D, Dixen U (2019) Recurrent event survival analysis predicts future risk of hospitalization in patients with paroxysmal and persistent atrial fibrillation. PLoS One 14:e0217983 CrossRef
go back to reference Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW (2010) Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology (Cambridge, Mass) 21:128 CrossRef Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW (2010) Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology (Cambridge, Mass) 21:128 CrossRef
go back to reference Van Oirbeek R, Lesaffre E (2016) Exploring the clustering effect of the frailty survival model by means of the brier score. Commun Stat-Simul Comput 45:3294–3306 MathSciNetCrossRef Van Oirbeek R, Lesaffre E (2016) Exploring the clustering effect of the frailty survival model by means of the brier score. Commun Stat-Simul Comput 45:3294–3306 MathSciNetCrossRef
Metadata
Title
Assessing model prediction performance for the expected cumulative number of recurrent events
Author
Olivier Bouaziz
Publication date
17-11-2023
Publisher
Springer US
Published in
Lifetime Data Analysis / Issue 1/2024
Print ISSN: 1380-7870
Electronic ISSN: 1572-9249
DOI
https://doi.org/10.1007/s10985-023-09610-x

Other articles of this Issue 1/2024

Lifetime Data Analysis 1/2024 Go to the issue

PREFACE

Preface