Causal effect estimation on restricted mean survival time under case-cohort design via propensity score stratification

The case-cohort design is a cost-effective approach used in large observational studies to investigate risk factors for survival outcomes (Prentice 1986). In this context, the “case" refers to subjects who develop the event during the study period. This design involves randomly selecting a subcohort from the full cohort, with unselected cases added to the subcohort to form the case-cohort. Covariates are only measured in the case-cohort. It is particularly useful when collecting covariate history for all individuals is impractical due to cost constraints, especially in large studies. The case-cohort design is especially advantageous when event rates are low, reducing data collection costs and time with minimal efficiency loss (Prentice 1986).

In practical applications, some covariates that are readily available for the full cohort, such as age, weight, and sex, may be correlated with the more expensive covariates. To make use of these fully observed covariates, a stratified case-cohort design was proposed, where the full cohort is stratified based on these covariates, and case-cohort sampling is carried out within each stratum (Borgan et al. 2000).

Analyzing case-cohort studies involves addressing the specific challenges rising from the case-cohort sampling as the ratio of case to non-case in the case-cohort is not representative of the full cohort. One class of methods focuses on the construction of a pseudo-likelihood function in which the risk sets are modified by certain weights to account for the case-cohort study design (Prentice 1986; Self and Prentice 1988; Kalbfleisch and Lawless 1988; Borgan et al. 2000; Kulich and Lin 2004). Another perspective views case-cohort studies as a missing-data problem, since subjects outside the subcohort have unobserved covariates, with the observation process controlled by study design. Marti and Chavance (2011) suggested using multiple imputation as an alternative to weighted analysis to handle this missing covariate information.

In large observational studies, it is often important to establish the causal relationship between a treatment or exposure and a survival outcome. There are many challenges to causal inference in observational survival data. First, standard statistical analysis methods are not sufficient to account for censoring associated with survival data. Second, while most studies measure the association with the hazard ratio (HR), it is not an appropriate measure of the marginal causal effect due to its conditional nature that induces selection bias (Hernán et al. 2004; Hernán 2010; Ni et al. 2021). Third, the Cox proportional hazards model is often used to estimate the HR but the proportional hazards assumption may not always be satisfied. Finally, observational survival data may suffer from confounding effects, as treatment groups often differ systematically in the distribution of the baseline covariates. Propensity score-based methods are commonly used to adjust for confounding effects. These methods include propensity score stratification (Rosenbaum and Rubin 1984; D’Agostino 1998; Cook and Goldman 1989; Lunceford and Davidian 2004; Austin 2011), matching (Rosenbaum and Rubin 1983; D’Agostino 1998; Rosenbaum et al. 2010; Stuart 2010; Zhao and Lu 2023), and weighting (Hahn 1998; Hirano and Imbens 2001; Hirano et al. 2003; Lunceford and Davidian 2004; Imbens 2004). In this paper, we employ propensity score stratification for confounding adjustment.

The difference in restricted mean survival time (RMST) is an alternative effect measure of the survival outcome to the HR. RMST, denoted as \(\mu (\tau )\), is the expected survival time during the first \(\tau \) follow-up period and is equivalent to the area under the survival curve up to \(\tau \) (Irwin 1949; Zhao and Tsiatis 1997; Andersen et al. 2004). Formally,where S(t) is the survival function. The truncation time point \(\tau \) is typically specified a priori based on subject knowledge. The treatment effect can be estimated as the difference in RMST between the treated and control arms. RMST difference is easy to interpret, collapsible, free of selection bias, and does not rely on the proportional hazards assumption (Ni et al. 2021; Lin et al. 2023). RMST does not suffer from selection bias because it is not a conditional quantity but rather an integration over follow-up time \(\tau \). RMST can be estimated nonparametrically by calculating the area under the Kaplan-Meier estimate of the survival function.

As mentioned in the previous section, propensity score stratification can be applied to remove confounding bias in observational studies. By stratifying on the propensity score, the covariate distribution between treated and control becomes similar within each propensity score stratum. Ni et al. (2021) proposed to use propensity score stratification to remove observed confounding bias and to estimate the RMST difference between treatment groups as a marginal causal effect measure. In this paper, we extend their method to the stratified case-cohort design so that causal effects can be estimated for large observational studies with survival outcome that utilize such a sampling design.

Coronary heart disease (CHD) is one of the leading causes of disability and death in the United States. High-sensitivity C-reactive protein (hs-CRP) is often associated with the risk of CHD (Katan and Luft 2018; Virani et al. 2021; Kochanek 2014). However, the causal relationship remains uncertain (Casas et al. 2008; Hingorani et al. 2009). The Atherosclerosis Risk in Communities (ARIC) study, an observational investigation involving 15,792 middle-aged US adults over several decades, aims to study the development of cardiovascular events and health-related issues (Ballantyne et al. 2004). An aspect of the ARIC study is to examine the association between inflammatory biomarkers such as hs-CRP and the risk of coronary heart disease. Due to the limited collection of plasma samples, a stratified case-cohort design was employed, stratified by age, sex, and race, with hs-CRP measured only in the selected case-cohort sample.

In an observational time-to-event study, suppose the research interest is in the causal effect of a binary treatment. Let A be the treatment indicator where \(A=0\) denotes control arm and \(A=1\) denotes treatment arm. Let X be a p-dimensional baseline covariate vector. Let \(T^A\) denote the potential event time under treatment A and \(S^A(t)\) be its survival function. With truncation time point \(\tau \), \(Z^A = \min (T^A, \tau )\) is the potential restricted event time. Let C be the random censoring time and \(T=AT^1 + (1-A)T^0\) be the observed event time. The observable quantities are U, \(\tilde{Z}\), and \(\delta _{\tilde{Z}}\), where \(U = \min (T,C)\) is the unrestricted observation time, \(\tilde{Z} = \min (Z,C)\) is the restricted observation time, and \(\delta _{\tilde{Z}} = I(Z<C)\) is the event indicator. Let \(N(t) = I(T\le t)\) be the counting process and \(Y(t) = I(T \ge t)\) be the at risk process. Under treatment assignment A, the potential restricted mean survival time is \(\mu ^A(\tau ) = \int _0^\tau S^A(t)dt\). Then the marginal causal effect of the treatment \(\nu \) is defined asWe require the following assumptions to estimate causal effects on RMST with propensity score stratification.

Note that Assumption 3 implies that \(C \perp Z^A \mid A\) since \(\tau \) is a constant, while Assumption 4 ensures asymptotically unbiased RMST estimates and nonzero variance estimates. In a stratified case-cohort study with B strata, the subcohort sampling probability in stratum b (\(b=1,...,B\)) is denoted as \(\alpha _{b}\), and we define \(\xi _{bi}\) as 1 if subject i in stratum b is included in the subcohort and 0 otherwise where \(i=1,\dots ,n_b\) and \(n_b\) is the size of stratum b in the subcohort. The case-cohort comprises all subjects from the subcohort and all cases outside the subcohort. In this paper, we use Bernoulli sampling to obtain the subcohort, where all \(\xi \)’s are independent. Some or all variables in X are observed only for members of the case cohort.

We propose, under a stratified case-cohort design, to use the RMST difference as the measure of marginal causal effect and to use propensity score stratification to remove confounding bias. The proposed method involves the following steps: stratified case-cohort sampling, propensity score estimation, propensity score stratification, and treatment effect estimation.

We generate six baseline covariates consisting of three continuous covariates, \(X_1\), \(X_2\), \(X_3\), and three binary covariates, \(X_4\), \(X_5\), \(X_6\). First, \(X_1\), \(X_2\), \(X^*_4\), \(X^*_5\) are generated from a multivariate normal distribution with mean of one, variance of 2, and pairwise covariance of 0.5. The two binary covariates, \(X_4\) and \(X_5\), are then obtained by dichotomizing \(X^*_4\) and \(X^*_5\) at one. \(X_3\) is defined as \(X_3 = 0.2 X_1 + 0.1 X_4 + 0.2 X_1 X_4 + \epsilon \) and \(X_6\) is defined by dichotomizing \(X^*_6 = 0.2 X_1 - 0.4 X_4 - 0.9 X_1 X_4 + \epsilon ^*\) at its median where \(\epsilon \) and \(\epsilon ^*\) are normally distributed with mean zero and variance of 0.5. The treatment indicator, A, is generated from a Bernoulli distribution where the treatment assignment probability, \(P(A=1)\), is specified as \(\textrm{logit}[P(A=1)] = 0.85 + 0.4 X_1 -0.4 X_2 + 0.3 X_3 + 0.4 X_4 - 0.4 X_5 - 0.4 X_6\). In this setting, approximately \(60 \%\) of the subjects receive treatment.

Independent failure times are generated from a proportional hazards model. Potential survival times \(T^A\) are generated from an exponential distribution with hazard \(\lambda (t) = \exp (0.5 + \beta _A A -0.5 X_1 + 0.5 X_2 + 0.4 X_3 -0.5 X_4 + 0.5 X_5 - 0.4 X_6)\). The censoring variable C is generated from an exponential distribution with hazard \(\theta (t) = \exp (\gamma -0.5A)\) and \(\gamma \) is specified to achieve the desired censoring rates. The true restricted event time Z, the unrestricted observation time U, and the restricted observation time \(\tilde{Z}\) are generated according to their definition. In this simulation, \(\tau \) is chosen to be the empirical 85th percentile of the unrestricted observation time U in a simulated dataset with an extremely large sample size, and \(\gamma \) is specified so that \(5\%\) of the subjects experience an event before \(\tau \) and an additional \(15\%\) of subjects who survive up to or beyond \(\tau \). Based on the modified definition of case (see Section 2.2.1), there are a total of \(20\%\) cases in the full cohort, corresponding to an 80% censoring rate.

After generating the full cohort, a stratified case-cohort sample is drawn from it. The full cohort is stratified by the binary variable \(X_4\) and stratum-specific case-cohort sampling probabilities are determined to achieve the desired non-case to case ratios in each stratum. Covariates \(X_1\), \(X_2\), \(X_4\), and \(X_5\) are completely observed in the full cohort while covariates \(X_3\) and \(X_6\) are only observed in the case-cohort. We consider scenarios where the treatment variable A is completely observed in the full cohort or is only observed in the case-cohort.

Two full cohort sample sizes (3000 and 10000), two treatment effect sizes in RMST (zero and \(17.76 \times 10^{-3}\)), and two non-case to case ratios (1 and 2) are considered. For non-case to case ratio of 1 and 2, \(40\%\) and \(60\%\) of subjects in the full cohort are included in the case-cohort, respectively. The truncation time \(\tau \) is set to 0.0614 and 0.1570 for the zero and nonzero treatment effect scenarios, respectively. We simulate 500 datasets for each simulation scenario. The true marginal treatment effect on RMST is determined by calculating the empirical difference between the potential RMST under treated and control from a large dataset of sample size. Both the true ATE and the true ATT are calculated as benchmarks for the methods examined in this simulation. The true ATE \(\varDelta _{ATE} = \sum _{i=1}^{n} \frac{1}{n}(Z_i^1 - Z_i^0)\). The true ATT \(\varDelta _{ATT} = \sum _{i=1}^{n} \frac{1}{n}I(A_i = 1)(Z_i^1 - Z_i^0)\).

In this simulation study, we compare the following causal effect estimation methods.

The performance of causal effect estimation is measured by bias, percent bias, model-based standard error (SE), empirical standard error (ESE), and \(95\%\) coverage probability (CP). The bias is defined as the estimated treatment effect minus the true effect. Percent bias is the bias of the estimate divided by the nonzero true treatment effect multiplied by 100. SE is the average of the estimated standard error based on the derived formula over all simulation replications. ESE is the standard deviation of the treatment effect estimates over all simulation replications. ESE is deemed as the true standard error. Finally, CP is the proportion of \(95 \%\) confidence intervals in all replications that contain the true treatment effect (Figs. 1 and 2).

The performance of estimating ATE for scenarios in which the treatment indicator is observed in the full cohort and only in the case-cohort is summarized in Figs. 4 and 2, respectively. Each figure contains scenarios with different treatment effects, sample sizes, and non-case to case ratios. Detailed numerical results on SE, ESE, and CP can be found in Table 1 and 2. Since the ATT estimation performance is similar to the ATE estimation performance, only the results of the ATE estimation are presented. The results of the ATT estimation can be found in Tables 5 and 6 in Appendix 1.4. Additional simulation results with 95% censoring rate can be found in Appendix 1.5.

Using the full cohort without propensity score stratification (Full Cohort), the estimated RMST difference is highly biased in all scenarios due to confounding. With propensity score stratification on the full cohort (Full Cohort PS), a majority of the bias is removed, indicating that propensity score stratification is effective in adjusting for confounding bias. However, the remaining bias in Full Cohort PS indicates the existence of residual confounding not accounted for by propensity score stratification. The proposed method based on the weighted Kaplan-Meier estimator for the stratified case-cohort sample (CC) performs similarly to Full Cohort PS in terms of bias, but with a larger ESE in all scenarios. The increased ESE in CC method is expected as it uses less data than Full Cohort PS. In both CC and Full Cohort PS, the SE is reasonably close to ESE and the CP is reasonably close to the nominal level considering the slightly biased point estimates due to residual confounding. Increasing the non-case to case ratio reduces the bias and improves the SE and CP of the CC method, especially for smaller sample sizes.

Full size image

Full size image

In scenarios where treatment is observed for all subjects, the correctly specified multiple imputation method (MI) has slightly smaller bias, similar CP, and smaller ESE than the CC method. The smaller ESE is due to the fact that MI makes better use of fully observed covariates than the CC method. When the multiple imputation model is misspecified (Mis MI) by omitting covariates \(X_2\) and \(X_5\), the bias increases and becomes greater than that in the CC method.

In the scenario where treatment is only observed for the case-cohort subjects, both MI and Mis MI have a much poorer performance than the CC method in all evaluation indices. This is likely due to the misclassification of imputed treatment during the imputation stage, which leads to biased propensity score estimates and in turn biased PS stratification. The performance of both MI and Mis MI substantially improves with increased non-case to case ratio since treatment needs to be imputed in fewer subjects, but their performance is still much worse than that of the CC method.

In summary, the proposed CC method gives the best overall causal effect estimation across all scenarios and therefore seems to be the most robust one to use when estimating causal effects on RMST based on case-cohort data. As the total sample size and non-case to case ratio increase, the precision of the estimated causal effects improves. Multiple imputation-based methods suffer from imputation model misspecification and biased propensity score estimation due to misclassification of imputed treatment.

To evaluate the impact of the number of PS strata on the performance of the proposed method, we performed additional simulations using 5, 7, and 10 PS strata. The results are included in Appendix 1.6. It seems that seven PS strata are adequate to remove most of the confounding bias. Any additional strata do not significantly reduce bias further.

Additional simulation results under an unstratified case-cohort design can be found in the Appendix 1.7. The performance of the proposed method is very similar to that under a stratified case-cohort design, confirming the applicability of the proposed method to both the unstratified and stratified case-cohort design.

The ARIC study monitored 15,792 individuals from four U.S. communities for cardiovascular events and mortality over several decades (Investigators 1989; Ballantyne et al. 2004). After excluding subjects with missing data, the full cohort includes 12,197 individuals. Those who survived and remained disease-free by the end of 1998, or who were lost to follow-up, were considered censored. A random subcohort of 978 individuals was selected through stratified random sampling based on sex, race (black vs. white) and age at baseline (\(\le 55\) years vs. \(>55\) years). The study documented 638 CHD cases or CHD-related deaths, resulting in a censoring rate of \(94.8\%\). After including all CHD cases, the case-cohort consisted of 1,567 subjects. The purpose of this analysis is to determine the average treatment effect (ATE) and average treatment effect on the treated (ATT) for individuals with hs-CRP levels \(>3\) compared to those with levels \(\le \) 3 with respect to RMST of CHD. To calculate the RMST for each hs-CRP group, a truncation time \(\tau \) of 8 years (2,920 days) was chosen, given the duration of follow-up. Table 3 presents summary statistics of the baseline variables, highlighting the imbalance of the covariate distribution between the high and low hs-CRP groups, which necessitates adjustment for confounding. RMST was estimated using the proposed weighted method and multiple imputation on the case-cohort data. Given the study design, multiple imputation was only necessary for the CRP level (high / low), as the CRP level was the only unobserved covariate in the full cohort. Additionally, the ARIC study was analyzed using the Cox proportional hazards model. All methods incorporated propensity score stratification with seven strata.

For weighted analysis, each individual in the subcohort was weighted by their case-cohort sampling probability. When applying MI, we imputed the CRP level using the Bayesian logistic regression (logreg) method found in the mice library in R (van Buuren and Groothuis-Oudshoorn 2011). Imputation for the CRP level was performed using information from all baseline characteristics, the log of observation time, and the event indicator. The propensity score was calculated using a weighted logistic regression on the case-cohort sample with all baseline covariates. The balance between low and high CRP groups within each propensity score stratum was examined. Ideally, the standardized mean difference for each covariate within each propensity score stratum should be less than 0.1. We observed an acceptable balance for most covariates within each propensity score stratum. However, we observed some imbalance with a standardized mean difference exceeding 0.1 between the treatment and control arms in certain variable-strata combinations for both the case-cohort and the imputed data. Achieving perfect balance within each propensity score stratum is challenging due to the coarse nature of propensity score stratification.

Based on the results presented in Table 4, the estimated ATE is much larger in absolute value in CC than in MI. The SE is larger in CC, which is consistent with the observation in the simulation study. However, as we also observed in the simulation study, MI estimates are likely to be biased. The \(95\%\) CI of the weighted method does not cover zero, indicating a significant difference in RMST between the lower and high CRP groups. The MI method did not find a significant difference in RMST. The estimated ATT and its SE are higher than the ATE for both methods. For ATT, neither method found a significant causal effect of high CRP on CHD RMST in those with high CRP. Finally, the stratified Cox proportional hazards model did not find a significant difference in terms of the hazard of CHD between the two CRP groups.

It is worth mentioning that the theoretical properties of the proposed method were established under the assumption of Bernoulli sampling, while the ARIC study uses simple random sampling. However, given that the random subcohort only represents about eight percent of the full cohort, the results from Bernoulli sampling and simple random sampling is expected to be very similar.

In this paper, we introduce propensity score stratified estimation of the marginal causal effect measured by the RMST difference under a stratified case-cohort design. RMST difference has the advantage of not relying on the proportional hazards assumption and allows for the estimation of the marginal treatment effect. We propose RMST difference as an alternative causal effect measure to the commonly used HR in epidemiological studies.

RMST and HR, while both applicable to survival outcomes, measure different aspects of treatment effects. Specifically, RMST measures the effects on the additive scale when HR measures the effects on the multiplicative scale. In rare disease studies, one may observe a large fold-change in the hazards (i.e., large HR) but a relatively small RMST difference. This suggests that while the treated group may have a much higher risk of disease than the control group in terms of fold change, it may not correspond to a large change in the expected survival time for the population studied. Therefore, the RMST difference provides complementary insights that are not discernible from HR alone. In addition, the RMST difference offers a more intuitive interpretation, making it easier for clinicians and patients to grasp, as it represents the difference in life expectancy over a specific time horizon.

It is important to note that the definition of cases in this paper differs slightly from that of the stratified case-cohort design proposed by Borgan et al. (2000). The new definition of cases is unique to using RMST as an effect measure, as it includes all subjects who survive up to \(\tau \) as cases, making better use of the information when estimating RMST. However, this new definition depends on the choice of \(\tau \). Thus, it may not be applicable when analyzing an existing case-cohort dataset where the case-cohort sample has already been drawn according to the conventional definition of case, as is the case in the ARIC study. Moreover, when \(\tau \) is small, there may be a large number of cases based on the new definition. In this case, a generalized case-cohort design could be considered, where only a random subset of unselected cases is included in the random subcohort to form the generalized case-cohort sample (Cai and Zeng 2007).

The independence between survival and censoring times in Assumption 3 is only conditioned on treatment A but not on covariates X. This is slightly stronger than the common assumption of independence conditional on A and X in the literature. This is because our proposed method stratifies subjects by propensity score rather than by covariates directly. Therefore, even if subjects have the same propensity scores, their covariate values can still be different. This difference may result in residual dependence between survival and censoring times within PS strata and thereby introduce bias in the RMST estimator. However, even if independence can only be achieved by conditioning on both A and X (thus violating Assumption 3), as the number of PS strata increases with the sample size, we expect the bias in the RMST estimator due to residual dependent censoring to become very small.

An important practical question is how to choose the truncation time \(\tau \). Ideally, \(\tau \) should be chosen by the investigator before the analysis based on clinical justification. Tian et al. (2020) proposed a data-driven approach to select \(\tau \), which could be adapted to a stratified case-cohort design.

In this paper, we proposed to estimate propensity scores using a weighted logistic regression to account for the case-cohort sampling design. Other machine methods such as bagging or boosting, random forests, and neural networks have been used to estimate propensity scores in the literature (Lee et al. 2010; Setoguchi et al. 2008; McCaffrey et al. 2004)5. It is an interesting future research topic to extend these methods to a stratified case-cohort design so that they can be used to estimate propensity scores in the settings considered in this paper.

One limitation of our approach is that propensity score stratification can only address observed confounding bias, leaving hidden bias from unobserved confounders unaccounted for. To mitigate this issue, sensitivity analysis methods, such as discussed in Rosenbaum et al. (2010), could be extended to a stratified case-cohort design. This presents another topic for future research.