A comparison of Kaplan–Meier-based inverse probability of censoring weighted regression methods

In some settings of survival analysis, the primary interest may concern one or a few time points, where measures of interest could be risk or restricted mean survival time. By focusing on one or a few time points, the analyst can easily communicate results. As pointed out by Martinussen and Scheike (2023), a particular time point may have a special role in a specific clinical subject area. For instance, disease-free survival 3 years after treatment for certain cancer patients may be taken to indicate cure of the patient. A relevant tool for adjusted comparisons of the measures of interest between groups in this sort of setting is a regression method such as logistic regression. The issue of right censoring becomes a missing data problem for such regression methods by potentially rendering the outcome unobserved. In the study of treatment effects on 3 year disease-free survival of cancer patients, such a right-censoring issue may, for instance, come about if it is desired to include in the analysis more recently treated patients for whom 3 years of follow-up is not yet available.

This paper is concerned with a regression setting and the handling of outcomes that may be missing due to right censoring. Here, the handling will in some way be by use of inverse probability weighting. The weights are related to the censoring distribution and for this reason such approaches may be considered inverse probability of censoring weighting (IPCW) approaches. On the other hand, this may be considered a misnomer since the weights are, much like sampling weights, estimates of the inverse probabilities of observing the outcomes in question rather than probabilities of censoring. This paper will consider three approaches where the weights are based on Kaplan–Meier estimates of the censoring distribution, either based on the entire sample or calculated in strata. An important assumption will be independent censoring.

One approach involves a Horvitz–Thompson-type weighting of the entire individual contribution to an estimating equation, which has been considered in a survival setting by e.g. Robins and Rotnitzky (1992). A second approach, which was suggested by Scheike et al. (2008) for assessing the influence of covariates on a cumulative incidence curve, involves weighting only the potentially missing outcome for each individual. The third approach that is considered here was introduced by Andersen et al. (2003) and involves replacing the potentially missing outcomes by jack-knife pseudo-values, here based on an inverse probability weighting estimator such as the Kaplan–Meier estimator. All three approaches can be carried out using standard statistical software in a wide range of situations: the software likely facilitates calculation of Kaplan–Meier estimates and regression parameter estimates in, for instance, generalized linear models, perhaps with sampling weights, as well as robust, Huber–White-type standard errors of the regression parameter estimates based on a standard sandwich variance estimator.

The paper by Blanche et al. (2023) studies and compares the first and second approach in a setting where the outcome is having a certain event within a certain time and where a logistic regression is considered. A main conclusion of that paper is that which method is more efficient depends on the censoring distribution. Another conclusion is that a naive approach to variance estimation will lead to too large, or conservative, variance estimates.

The third approach, also known as the pseudo-observation method, is reviewed by Andersen and Pohar Perme (2010) where some further background can be found. Much of the theory on the method that is useful for the purposes of this paper can be found in Overgaard et al. (2019). The paper by Andersen and Pohar Perme (2010) also suggests stratifying calculation on a variable if censoring depends on that variable, which is an approach studied in more detail in this paper.

Although the theory of these three different approaches seems clear, at least in some settings, it is, however, not clear how the three approaches compare, especially how the pseudo-observation method compares with the two other approaches in terms of efficiency. Results by Binder et al. (2014) and Parner et al. (2023) indicate that the pseudo-observation method makes more efficient use of data compared to the outcome weighting approach, but it remains unclear why this is the case and how generally this holds.

In this paper, a comparison of the three approaches is carried out theoretically, in a theoretical example, and in simulations. The theoretical comparison is in terms of the asymptotics of the three approaches in a general setting which is laid out within a common framework that includes stratification of the weight calculation. The topic of naive variance estimation using the standard sandwich variance estimator is considered for the three approaches. Some insights into the biases of the three approaches when the independent censoring assumption is violated are also gained.

In Sect. 2, the setting and main results on, in particular, the asymptotic variances are presented. In Sect. 3, a theoretical example offers insights into how the three approaches compare in a specific setting. In Sect. 4, simulations are used to illustrate, corroborate, and challenge the asymptotic results. The paper ends with a discussion of results in Sect. 5 and an appendix with technical results in section A.

Suppose Y is an outcome of interest and a model is assumed which states that a parameter vector \(\beta \in \mathbb {R}^p\) exists such that \(\operatorname {E}(Y \mathbin {\mid }X) = \mu (\beta ; X)\) for a certain function \(\mu \) where X is a vector of covariates. It is desired to estimate the true \(\beta \) based on n observations, i.e. n independent replications of that experiment. A standard approach would be to solve an estimating equation of the typefor a suitable p-dimensional vector function A. When Y is not always observed due to censoring, this approach can no longer be taken. To be more specific, a competing risks setting is now considered. Suppose Y is a function of a failure time \(T > 0\) and a failure type \(D \in \{1, \dots , d\}\), but determined by a time point \(t>0\). In other words \(Y = y(T \wedge t, D \mathbbm {1}(T \le t))\) for a reasonable function y. Outcomes of this type includeMore generally, T may be some event time and D an event type. Possible models of \(\operatorname {E}(Y\mathbin {\mid }X)\) include examples from generalized linear models: a linear model \(\mu (\beta ; x) = \beta ^{\textsf{T}}x\); a relative or exponential model, \(\mu (\beta ; x) = \exp (\beta ^{\textsf{T}}x)\); or a logistic model \(\mu (\beta ; x) = 1/(1+\exp (-\beta ^{\textsf{T}}x))\). The logistic model is primarily appropriate for the dichotomous outcomes. In all examples, the vector x likely includes a constant term and should more generally be considered a result of a vector function applied to an original set of covariates since it should also be able to hold interaction terms, for instance. In the generalized linear model setting, the function A depends on the choice of link function, which determines the structure of \(\mu \), and on the choice of family. The Gaussian family corresponds to the choice of \(A(\beta ;x) = \frac{\partial }{\partial \beta } \mu (\beta ;x)\). A simple choice is \(A(\beta ;x) = x\) and this is obtained when the link function used is canonical for the specified family.

Next, suppose T and D are not always observed due to censoring at a censoring time \(C > 0\). Instead \(\tilde{T} = T \wedge C\), the observed exit time, and \(\tilde{D} = D \mathbbm {1}(T \le C)\), the observed exit type with 0 denoting censoring, are observed. An outcome available at time t as above is observed if \(C \ge T \wedge t\) and can in this case be written as \(Y = y(\tilde{T} \wedge t, \tilde{D} \mathbbm {1}(\tilde{T} \le t))\). To handle censoring, different weighting approaches are considered. The weights are allowed to depend on covariates only through a categorization Z of the covariates X. If G denotes the conditional survival function of the censoring time C, that is, \(G(s \mathbin {\mid }z) = \operatorname {P}(C > s \mathbin {\mid }Z = z)\), a weight to consider is thenThe exact G may well be unknown, and here an approach where an estimate of the censoring distribution is used instead is considered. In the following, estimation based on n independent replications of this type of experiment where information on \((\tilde{T}, \tilde{D}, X)\) is available is considered. This setting, where censoring times are also censored by event times, allows for the use of the Kaplan–Meier estimator of G within each stratum of Z. In order to handle ties systematically and appropriately according to the described setting where the event time T takes priority over the censoring time C, a slight variation of the Kaplan–Meier estimator will in fact be used and is defined precisely in the appendix in equation (38). This estimate is called \(\hat{G}\). So, the applied weight will instead beThree regression approaches based on the weights \(\hat{W}\) are now considered: weighting the individual contribution and solvingweighting only the potentially censored outcome and solvingand replacing the outcome with a jack-knife pseudo-observation, \(\hat{\theta }_i\), from a weighted estimator and solvingThe pseudo-observation will be defined aswhere \(\hat{\theta }\) is the overall estimate of \(\operatorname {E}(Y)\),and \(\hat{\theta }^{(i)}\) is the estimate obtained by leaving out observation i, which can be writtenif \(\hat{G}^{(i)}\) is the stratified Kaplan–Meier estimate of G where observation i has been left out. Since \(\hat{G}^{(i)}(s \mathbin {\mid }z) = \hat{G}(s \mathbin {\mid }z)\) when observation i does not belong to stratum z, the jack-knife pseudo-observation of the overall estimator of (12) can also be calculated as the jack-knife pseudo-observation of the within-stratum estimate and equalsIn other words, the pseudo-observation includes the weighted outcome from before and an additional term that takes a potential influence of the observation on the estimate of the weight into account.

It may be noted that the inverse probability weighted estimates of (12) include as examples the Kaplan–Meier estimate for the outcome \(Y = \mathbbm {1}(T > t)\), the area under the Kaplan–Meier curve for the outcome \(Y = T \wedge t\), and other common estimators of the mentioned examples of outcomes Y. Satten and Datta (2001) established this type of result for the Kaplan–Meier-based estimate of a failure probability.

Generally, the following assumptions on the censoring mechanism are made.

The assumptions ensure that W is well defined and that \(\operatorname {E}(W \mathbin {\mid }T, D, X) = 1\), which make the weights suitable for compensating for the missing information. The cumulative censoring hazard can also be defined without issue in the relevant interval as follows. Let \(F_0(s \mathbin {\mid }z) = \operatorname {P}(C \le s \mathbin {\mid }Z=z)\). Define the corresponding cumulative censoring hazard by \(\Lambda (s \mathbin {\mid }z) = \int _0^s G(u- \mathbin {\mid }z)^{-1} F_0(\textrm{d} \,u \mathbin {\mid }z)\) for \(s \le t\). No assumption of continuity of these functions is made.

Other assumptions are made in the following. These assumptions include some usual requirements for the estimating procedure of the uncensored case in (1) to work, and a further positivity assumption, say \(\operatorname {P}(T> t \mathbin {\mid }Z) > 0\) almost surely, to ease the handling of the estimation of the censoring distribution. To be vague, these assumptions are collectively termed regularity conditions in what follows. Some further details on this matter are given in the appendix. The approach presented in the appendix even puts restrictive assumptions on the outcome Y, in particular boundedness. These assumptions are met by the four presented examples of outcomes, but do not seem strictly necessary for the results presented in the following.

Under regularity conditions, the original estimating equation, (1), based on uncensored information has, with a high probability for large n, as solutions consistent and asymptotically normally distributed estimates of the true \(\beta \), which will be denoted \(\beta _0\) in the following. This can be seen from results on Z-estimators, see for instance Chapter 5 of van der Vaart (1998). To be more specific, the estimates will be asymptotically linear with influence functionwhere \(B(X) = J(\beta _0)^{-1} A(\beta _0; X)\) and \(J(\beta ) = \operatorname {E}( - A(\beta ; X) \frac{\partial }{\partial \beta } \mu (\beta ; X))\). This means thatso that the asymptotic variance is \(\operatorname {Var}(\dot{\beta }(T, D, X))\).

Under similar regularity conditions, and the assumptions on the censoring mechanism mentioned above, the three weighting approaches similarly have as solutions estimates that are asymptotically linear with a specific influence function. The influence functions compare to the influence function of the uncensored problem in a certain way, as laid out in the theorem below. The notation \(M(s \mathbin {\mid }Z) = \mathbbm {1}(C \le s) - \int _0^s \mathbbm {1}(C \ge u) \Lambda (\textrm{d} \,u \mathbin {\mid }Z)\) will be used for a martingale related to the censoring.

The proof can be found in the appendix.

The last term of (17) is structured with a part depending only on the underlying competing risks data and its distribution and a part depending only on the underlying censoring time and its distribution. Under the assumptions, this structure implies a similar structure of the resulting asymptotic variance matrices which makes clear how the variance has been increased by censoring and how the variance depends on the censoring distribution.

With the asymptotic variances at hand, it is of interest to consider the question of which of the types has the lower asymptotic variance and can therefore be expected to produce the lowest variance in parameter estimates, at least at larger sample sizes. The answer clearly depends on many things. In particular it depends on how the three \(\Phi _type (s \mathbin {\mid }Z)\) compare at various time points s, but also on the censoring hazard at these time points. It is worth pointing out that there is a lower bound to what can be achieved in terms of asymptotic variance of these three approaches. Note how \(\phi _type \) is on the form \(B(X)(Y-f(X))\) for some function f of the covariates for each of the types of approach. For a given s this form hasby the law of total variance. This implies that \(f(x) = \operatorname {E}(Y \mathbin {\mid }T > s, X=x)\) minimizes this conditional variance over functions on this form. In other words, let \(\Phi (s \mathbin {\mid }Z) = \operatorname {Var}(B(X)(Y- \operatorname {E}(Y \mathbin {\mid }T> s, X)) \mathbin {\mid }T > s, Z)\), then \(\Phi _type (s \mathbin {\mid }Z) \ge \Phi (s \mathbin {\mid }Z)\) for each type. The variance expression of (21) now reveals how a comparably low variance among the three different types can be achieved for a type of approach that has \(\Phi _type (s \mathbin {\mid }Z)\) close to \(\Phi (s \mathbin {\mid }Z)\) at time points s where the censoring hazard is high.

Overall, the pseudo-observation approach may seem to have the best chance of producing a low variance, but these are of course rather vague observations. An example corroborating this observation is given in Sect. 3.

The outcome of failure is considered in detail by Blanche et al. (2023) and their Corollary 2 states that the difference in asymptotic variance matrix between the individual weighting approach and the outcome weighting approach can be negative definite or positive definite depending on the censoring distribution, which is in line with the observations above. It seems that this conclusion cannot immediately be transferred to, to give an example, the outcome of restricted survival time since \(\operatorname {E}(Y \mathbin {\mid }T > s, X)\) for s approaching t would be close to t rather than 0 in this case. As alluded to above, a similar conclusion can be reached in the comparison of the individual weighting approach and the pseudo-observation approach even for an outcome such as the restricted survival time.

A standard approach to estimating the variance of parameter estimates is to use the corresponding Huber–White-type sandwich variance estimator. For each type, \(U_{n,type }(\beta )\) is on the form \(\sum _{i=1}^n u_{i,n,type }(\beta )\). The corresponding standard sandwich estimate of the asymptotic variance isevaluated at the corresponding \(\beta \) estimate. This or a similar variance estimate is what tends to be used for producing standard errors of \(\beta \) estimates by statistical program packages when sampling weights are used or when robust standard errors are requested in regression procedures. It is of interest to establish results on the usefulness of this standard approach to variance estimation in the present setting. The next result establishes that the standard sandwich variance estimate will be conservative for large n for all three approaches when the model holds and it thereby extends Proposition 1 of Blanche et al. (2023).

The proof of the theorem can be found in the appendix.

The stated results assume that the regression model \(E(Y \mathbin {\mid }X) = \mu (\beta ; X)\) holds for some \(\beta \), also denoted \(\beta _0\). As can be seen from the results of the appendix, Lemma 5 and Lemma 6 specifically, the results do not change much under misspecification of the regression model. The three approaches will be able to estimate a best fit to the uncensored problem which may be useful in some situations. The best fit will then depend on the covariate distribution. Under misspecification of the regression model, the only real differences to the results stated earlier are that \(\beta _0\) should now refer to the best fit rather than a true \(\beta \) and that \(J(\beta _0)\) and thereby B(X) will have a more complicated expression, which can be found in equation (88) of the appendix. This does not change the observations made above much except perhaps for one observation concerning the individual weighting approach: The \(\operatorname {E}(Y \mathbin {\mid }T > s, X) \) can no longer be expected to be as close to \(\mu (\beta _0; X)\) as before for s close to 0 and so \(\Phi _\textit{ind}(s \mathbin {\mid }Z)\) can no longer be expected to be as close to the lower bound \(\Phi (s \mathbin {\mid }Z)\) as before. Concerning variance estimation, statistical program packages may or may not use an appropriate estimate of the more general and complicated expression of \(J(\beta _0)\) relevant under misspecification of the regression model when producing the standard sandwich variance estimate. If they do, Theorem 2 will apply with the mentioned modifications. If they do not, there will be another source of bias of the variance estimate, which will not be examined more closely here. The discussion concerning the more complicated and the more simple expression of \(J(\beta _0)\) relates to the discussion of whether to use the observed information matrix or an estimate of the expected information matrix under the model.

A source of bias in the regression parameter estimates is violation of the independent censoring assumption. An attempt at examining the bias is made in Proposition 1 of the appendix where a first order approximation of the bias is found for the three approaches. In principle, this gives some insights into how the bias compares between the three approaches, but the result is approximate and the conclusion is not particularly clear. One observation concerns the scenario where but not holds. Here, Eq. (82) of the appendix reveals how discrepancy between the true and fitted censoring hazard, according to the approximation, Overall, it seems the pseudo-observation approach will have the best chance of mitigating bias from this sort of violation of the independent censoring assumption by choosing Z to be predictive of the outcome as well as the censoring.

With an aim of comparing the asymptotic variances precisely, an example in a simple setting is now considered.

Suppose covariates consists of two groups expected to be equal in size, \(\operatorname {P}(X=0) = \operatorname {P}(X=1) = 1/2\). The event time is chosen to follow uniform distributions in the two groups,for certain choices of \(p, q \in (0,1)\). The outcome of interest is \(Y = \mathbbm {1}(T \le 1)\), corresponding to a time point of interest \(t=1\). A true model is \(\mu (\beta ; X) = \beta _0 + \beta _1 X\) with \(\beta _0 = q\) and \(\beta _1 = p-q\). Censoring occurs with probability 0.5 at a specific time point s only and this happens independently of other variables.

The three approaches are used for estimation of the \(\beta \) parameters with the choice \(A(\beta ; X)=(1, X)^{\textsf{T}}\). No stratification will be used. The asymptotic variances for the three types arein this case according to Corollary 1. Differences in asymptotic variances, particularly the signs of differences, are in this way given from differences in \(\Phi _type (s)\).

The matrix \(J(\beta )\) issuch thatCalculations revealThe asymptotic variance of the \(\beta _1\) component is considered by applying the vector \(a = (0, 1)^{\textsf{T}}\). Calculations reveal, for a general a,For the particular choice of vector \(a = (0, 1)^{\textsf{T}}\),and therebywhich is negative for all \(s \in (0,1)\), indicating an asymptotic advantage of the pseudo-observation approach over the outcome weighting approach in this setting. Also,Certainly for \(s < \min (\frac{1}{2-p},\frac{1}{2-q})\) the difference is negative, in favor of the individual weighting approach over the outcome weighting approach, while for \(s > \max (\frac{1}{2-p},\frac{1}{2-q})\) the difference is positive, in favor of the outcome weighting approach.

The comparison of the pseudo-observation approach and the individual weighting approach may be obtained by subtracting the two expressions from each other. The difference is in favor of the pseudo-observation approach for \(s=1\) and is 0 for \(s=0\). Seemingly, the difference for any \(s \in (0,1)\) will be an intermediate value and so in favor of the pseudo-observation approach over the individual weighting approach.

For the estimation of the intercept parameter, \(\beta _0\), the vector \(a= (1, 0) ^{\textsf{T}}\) is instead considered. With this choice,which leads toThis is in favor of the pseudo-observation approach over the outcome weighting approach when and only when \((p+q)(1-qs) < 2q (2-ps-qs)\) which happens when \(s < \frac{3q - p}{q(p+q)}\). That can happen potentially never and potentially always in the interval (0, 1) depending on the values of p and q. Next,which is in favor of the individual weighting approach when \(s(2-q)-1 < 0\), that is, when \(s < \frac{1}{2-q}\). With this result, examples can be found where the asymptotic variance of the \(\beta _0\) estimate is smaller for the individual weighting approach than for the pseudo-observations approach. Take for instance \(p=1/2\), \(q=1/6\) and \(s \in (0, 1/(2-\frac{1}{6}) )\).

This simple theoretical example illustrates how examples can be found where either of the three approaches will have the smallest asymptotic variance.

To gain insights into the behavior of the three approaches in finite samples, a simulation study has been conducted as described in the following. Three separate scenarios are considered in this simulation study, each with different configurations. Each configuration is simulated 10,000 times.

Scenario I—cumulative incidence. This scenario considers the comparison of the cumulative incidence, or risk, in two groups by a risk difference. The purpose is a comparison of the three approaches in a simple setting at different censoring distributions and various sample sizes. The simple setting corresponds to the theoretical example given in Sect. 3. First, the group \(X \in \{0,1\}\) is drawn with equal probability \(\operatorname {P}(X=x) = 0.5\) in the two groups. The time to event, T, is drawn according to \(\operatorname {P}(T \le s \mathbin {\mid }X=x) = p_x s\) for \(s \in (0, 1/p_x)\) where the choices \(p_0 = 1/6\) and \(p_1 = 1/2\) are used. The time point \(t=1\) is the time point of interest in the following, and the outcome \(Y = \mathbbm {1}(T \le 1)\) is the outcome of interest. A simple linear model, \(\mu (\beta ; X) = \beta _0 + \beta _1 X\) is considered such that \(\beta _1\) is the risk difference and the unknown parameter to be estimated. The specification above makes \(\beta _1 = p_1 - p_0 = -1/3\) the true value of the risk difference at the time point of interest. The censoring distribution will be independent of T and X. Three censoring distributions are considered: one where about \({50}\%\) are censored at the early time point 0.2, \(P(C = 0.2) = 0.5\), and the rest remain uncensored; one where about \({50}\%\) are censored at the late time point 0.8, \(P(C = 0.8) = 0.5\), and the rest remain uncensored; and an exponential censoring distribution, \(\operatorname {P}(C > s) = \exp (-s)\). Samples of sizes \(n \in \{50, 100, 200, 400, 800\}\) are considered. The three approaches are used unstratified, that is, the overall Kaplan–Meier estimator of the censoring distribution is used for the weights, and with \(A(\beta ; X) = (1, X)^{\textsf{T}}\). Note that the choice \(\operatorname {P}(C = s) = 0.5\) and the rest remaining uncensored, or \(\operatorname {P}(C \ge 1) = 0.5\), will have \(\Delta \Lambda (s)/G(s) = 0.5/0.5 = 1\) and is able to pick out the remaining integrand in the last part of (21) at the chosen time point s, for instance \(s=0.2\) or \(s=0.8\). In other words, \(\Sigma _type = \Sigma + \Phi _type (s) S(s)\) in this case.

Full size image

Key results of the simulations in this scenario are summarized in Fig. 1 and Table 1, where the focus is on the variance of \(\beta _1\) estimates. Observed biases for \(\beta _1\) in the approaches seem negligible and are not presented here. As is illustrated in Fig. 1, the observed variance of the parameter estimates is in line with what is suggested by the theory and the theoretical example of Sect. 3, especially for the larger sample sizes: the pseudo-observation approach produces a comparably low variance in this setting in all three censoring distribution configurations; the individual weighting approach produces a comparably low variance when censoring occurs at an early time point; the outcome weighting approach produces a reasonably low variance when censoring occurs at a late time point. It can also be seen how the individual weighting approach eventually produces a lower variance than the outcome weighting approach in the exponential censoring distribution configuration in this setting, which is not immediately clear from the theory already presented. The fact that losing information to censoring results in larger variances of parameter estimates, as seen in Corollary 1, is illustrated by the gap from the observed variances of the three approaches to the observed variance of the underlying approach based on uncensored data. It seems reasonable that the gap is smaller when censoring occurs at a late time point since less information is lost.

Table 1 gives further insights into the variance estimation using the standard, Huber–White-type sandwich variance estimator: In most cases the average variance estimate is larger than the observed variance in parameter estimates, as suggested by Theorem 2. Somewhat surprisingly, the opposite does happen even for larger samples sizes, at least for the individual weighting approach. A calculation reveals that the asymptotic difference is quite small in this case and it can apparently not be expected to show up in simulations at this sample size and number of iterations. Overall, the differences between observed variance and average variance estimate are fairly small, and the corresponding coverage probabilities of Wald-type \({95}\%\) confidence intervals using these variance estimates are quite close to \({95}\%\), at least for the larger sample sizes. At lower sample sizes, some degree of underperformance in terms of coverage probability is seen, particularly for the individual weighting approach.

To sum up this scenario, the pseudo-observation approach generally wins out in terms of producing low variance of the estimates of the important regression parameter in this case, and the standard sandwich variance estimator even produces reasonable variance estimates for the pseudo-observation approach, leading to reasonable coverage probabilities.

Scenario II - restricted mean, a misspecified model, and covariate-dependent censoring. In this scenario, the restricted mean for different values of continuous covariates are compared using differences. The purpose is a comparison of the three approaches with a new outcome type in a more complicated situation where there are more covariates, continuous covariates, misspecification of the regression model, and covariate-dependent censoring. Three independent continuous covariates are considered, \(X=(X_1, X_2, X_3)\) where \(X_1 \sim N(0,1), X_2 \sim U(0,1), X_3 \sim \Gamma (\text {shape } 3, \text {scale } 0.5)\). Given X, the event time T will follow a Weibull distribution with shape parameter 1.5 and rate parameter \(\exp (-2 + X_1 + X_2/6 + X_3/2 + X_2 \cdot X_3/4)^{\frac{1}{1.5}}\). The time point of interest is \(t=1\) and the outcome of interest is then \(Y = T \wedge 1\). The model considered is \(\mu (\beta ; X) = \beta _0 + \beta _1 X_1 + \beta _2 X_2 + \beta _3 X_3\). No attempt will be made to find \(\operatorname {E}(Y \mathbin {\mid }X)\), but it is apparent that \(\mu (\beta ; X)\) is misspecified. Given X, the censoring distribution will be independent of T and follow a Weibull distribution with shape parameter 1.5 and rate parameter \(\exp (-0.5 + X_2)^{\frac{1}{1.5}}\). In particular, the censoring distribution depends on the uniformly distributed \(X_2\). The three approaches are used with \(A(\beta ; X) = (1, X_1, X_2, X_3)^{\textsf{T}}\) and the sample size \(n=1000\) is considered throughout. It is expected that handling the covariate-dependent censoring by stratification will be useful. The stratification variable Z is constructed from \(X_2\) by first choosing a number of strata k, then letting \(Z = j\) when \(j/k < X_2 \le (j+1)/k\) for \(j = 0, \dots , k-1\). The three approaches are considered with \(k \in \{1, 2, 4, 8\}\). The conditional censoring hazard rates are proportional with proportionality factor \(\exp (X_2)\). This implies that the maximal conditional censoring hazard rate differs from the minimal by a factor of \(\exp (1/k)\) within stratum when k strata are considered. These factors are 2.72, 1.65, 1.28, and 1.13 for the considered k. Note that although the model is misspecified, the fits from the three approaches should on average match the best fit from uncensored problem if the censoring mechanism is handled appropriately according to the theory.

Full size image

Full size image

As can be seen from Fig. 2, all three approaches produce parameter estimates that on average resemble the average parameter estimates from the uncensored data when a large degree of stratification is used. The level of resemblance increases with the number of strata. At a low number of strata, the outcome weighting approach produces the worst resemblance and the pseudo-observation approach the best resemblance in this setting. This is in line with an earlier observation that the pseudo-observation approach may have the best chance of having a limit close to the best fit in uncensored data.

The results on variance and variance estimation are presented in Fig. 3. Notably, the outcome weighting approach produces a very large variance in the \(\beta _2\) estimates, at least at a low number of strata, in comparison with the other approaches. As suggested by an earlier observation, the variance in these parameter estimates for the outcome approach does decrease with the number of strata used in the estimation of the censoring distribution. As also suggested by an earlier observation, this decrease is not seen for corresponding standard sandwich variance estimates. A similar pattern is seen for the individual weighting approach for the \(\beta _2\) parameter, but at a much lower level of variance. Overall, the pseudo-observation approach produces the smallest variances of parameter estimates and these observed variances seem well-reflected by corresponding average variance estimates. Perhaps surprisingly, the observed variances almost seem to tend to increase with the number of strata, but with the scale in mind, it must also be fair to claim that the observed variances do not change much with the number of strata. There are similarly many apparent deviations from earlier theoretical and asymptotic observations: Observed variances increasing with the number of strata for the individual and outcome weighting approaches; variance estimates that are on average smaller than the observed variances. It is worth noting that some of these theoretical observations were made under the assumption of a true regression model and independent censoring in strata, which does not hold in this example.

In conclusion, this scenario demonstrates a setting with a total failure of the outcome weighting approach in terms of variance and also the failure of the standard sandwich variance estimator, at least for the individual and outcome weighting approaches.

Scenario III - risk ratios and stratifications in a factorial design. A \(2^5\) factorial design is considered in this scenario with a complete symmetry in the 5 factors. Focus will be on estimation of the risk ratio related to one factor while taking the other factors into account. The purpose is a comparison of the three approaches in low sample sizes and with a potentially heavy degree of stratification. There is a potential for \(2^5 = 32\) strata. Let a stratum be given by \(x = (x_1, x_2, \dots , x_5)\) where \(x_1, \dots , x_5 \in \{0,1\}\) denote the 5 factors. In such a given stratum, T is drawn from a uniform distribution on \((0, 1/(0.1 \cdot 1.25^{x_1 + x_2 + \cdots + x_5}))\), and the outcome \(Y = \mathbbm {1}(T \le 1)\) is considered. The expectation is \(\operatorname {E}(Y \mathbin {\mid }X=x) = 0.1 \cdot 1.25^{x_1 + \cdots + x_5}\). For the estimation procedure, consider the model specified by \(\mu (\beta ; x) = \exp (\beta _0 + \beta _1 \cdot x_1 + \cdots + \beta _5 \cdot x_5)\). This means the true values have \(\exp (\beta _j) = 1.25\) for \(j=1, \dots , 5\). The parameter \(\beta _1\) is considered of primary interest. The censoring time is drawn, independently from T, from a uniform distribution on (0, 5/3). Scenarios with 2, 6, and 12 observations per stratum are considered. In other words, \(n \in \{64, 192, 384\}\). The three approaches are used with \(A(\beta ; X) = \frac{\partial }{\partial \beta } \mu (\beta ; X)\) and stratification on either no factors, the first factor, the first three factors, and on all five factors. Note that the leave-one-out calculations of the pseudo-observation approach are carried out at their lower limit of sample size in the case of stratification on all factors and only 2 observations per stratum.

The results of Scenario III are summarized in Table 2. Due to possible non-convergence for this estimation problem at small sample sizes, the convergence percentage is reported. The remaining statistics concern the select replications where convergence is achieved for all three approaches. Convergence is required to happen in 20 iterations. Due to possible outliers of parameter estimates and variance estimates, the median variance estimate scaled by n and the scaled variance expression \(\frac{n}{\Phi ^{-1}(3/4)^2} \operatorname {MAD}^2\) based on the median absolute deviation of parameter estimates, \(\operatorname {MAD}\), and standard normal cumulative distribution function, \(\Phi \), are presented as robust alternatives to observed means and variances.

The individual weighting approach is seen to have the most convergence problems in small sample sizes in this setting. The coverage probabilities are generally too low for the three approaches at the small sample sizes, while they are slightly too large at the largest sample size. The pseudo-observation approach seems to have the largest problems in terms of coverage with a large degree of stratification at lower sample sizes. As suggested by the coverage probabilities, the summary statistics for variance and variance estimation can be quite off in settings with small sample sizes. At the largest sample size, the variance estimate summary is larger than the variance expression for the observed parameter estimates, which is in line with the theoretical observations made earlier. At the largest sample size and the lower degree of stratification, the pseudo-observation approach is producing the lowest numbers in this setting, whereas the outcome weighting approach produces the lowest numbers at a larger degree of stratification. Overall, nothing seems to be gained in terms of variance for either of the three approaches by applying a larger degree of stratification in this setting.

As a conclusion, this scenario has certainly crossed the line into territory where neither of these approaches work well. The scenario gives an example where the individual weighting approach seems more likely not to achieve convergence, and an example where the pseudo-observation approach is more vulnerable to too much stratification in the estimation of the censoring distribution. It also gives an example of a setting where not much is gained by applying stratification.

This paper has extended some of the results presented in Sect. 2 of Blanche et al. (2023): In addition to the individual and outcome weighting approaches, called IPCW-GLM and OIPCW by Blanche et al. (2023) respectively, the pseudo-observation approach is now also considered; the type of outcome and type of model is now more general and not restricted to the logistic model; more results on the consequences of stratification in the estimation of the censoring distribution are given; and the presented results do not depend on continuity of the involved distributions, which may be useful when emulating expressions empirically. A main point is also an extension of a result from Blanche et al. (2023): Neither of the three approaches will generally have the lowest asymptotic variance. Which of the approaches will produce the lowest variance of parameter estimates will depend on the setting. The expressions given in this paper should help judging which one will win out. If censoring primarily happens early on in the time interval, this should favor the individual weighting approach, while if censoring primarily happens late in the time interval, this should favor the outcome weighting and the pseudo-observation approaches. Theoretical observations and simulations in this paper suggest the pseudo-observation approach may tend to have the most advantages such as smaller variance of parameter estimates and smaller bias from a misspecified censoring distribution. If it is of interest to use the standard sandwich variance estimator, this variance estimator also seems to be most appropriate for the pseudo-observation approach. On the other hand, the pseudo-observation approach can be expected to be the most time-consuming approach, and it has also been seen to work less well when only few observations are available per stratum. Additionally, the simulation settings used in this paper may favor the pseudo-observation approach by design by having a larger censoring rate, or rather \( \Lambda (\textrm{d} \,s)/G(s)\), at later time points rather than earlier time points. This may be realistic, but is of course not guaranteed in applications. One simulation scenario saw the outcome weighting approach producing a tremendously large variance in parameter estimates in a setting concerning restricted event time, \(T \wedge t\). The theory would suggest this happens because \(\operatorname {Var}(B(X) (T \wedge t) \mathbin {\mid }T > s, Z)\) is large for s close to t. Arguably, this is a problem with the outcome definition rather than the outcome weighting approach. A similar model could be studied by focusing on, and weighting, the outcome \(t - T \wedge t\), the time lost before t. The model could, in principle, be changed along the same lines to \(\tilde{\mu }(\beta ; X) = t - \mu (\beta ; X)\) without changing the uncensored problem. With this outcome, \(\operatorname {Var}(B(X) (t - T \wedge t) \mathbin {\mid }T > s, Z)\) would be small for s close to t, and is expected to resolve the issue although this was not pursued here. This issue did not arise in Scheike et al. (2008) where focus was on risk, but it may be worthwhile having in mind when using the outcome weighting approach with other types of outcomes.

There are indications in the theory that stratification should be helpful for reducing bias in regression parameter estimates from covariate-dependent censoring and that a high degree of stratification may help reduce the asymptotic variance even when independent censoring is achieved at a lower degree of stratification. The simulation scenario II, however, demonstrates how reducing the bias by increasing the degree of stratification may come at the cost of a larger variance in regression parameter estimates. For instance, estimation of the parameter \(\beta _2\) may preferably be carried out by using the pseudo-observation approach with a low degree of stratification in that scenario if a low mean squared error is desired relative to the best fit in the uncensored problem. It is then very difficult to provide any general practical guidance on stratification granularity based on the results presented here. In the specific setting of simulation scenario II it may be worth considering that the within-stratum maximal to minimal censoring hazard ratio was 2.72 for one stratum, 1.65 for two strata, 1.28 for four strata, and 1.13 for eight strata. If the resemblance of estimates to the best fit of the uncensored problem is deemed to be sufficient at, for instance, two strata and four strata for the individual weighting approach and the outcome weighting approach respectively, these two choices would then correspond to a maximal within-strata relative difference in censoring hazard rates of \({65}\%\) and \({28}\%\), respectively, in that scenario. On the other hand, simulation scenario III gives an example with \(n=192\) and five parameters to be estimated where performance in terms of coverage probabilities seems to drop markedly between \(6 \cdot 2^{5-3} = 24\) observations per stratum and 6 observations per stratum for the individual weighting approach and the pseudo-observation approach. In other words, there is certainly a limit to how few observations per stratum it is possible to work with.

A theme of this paper has been the bias of the standard sandwich variance estimator as seen in Theorem 2. The bias is upwards under the assumptions and leads to conservative inference if confidence intervals and statistical tests are based on this variance estimator. This applies to all three approaches. In simulation scenario II, the variance estimate was seen to be much too large for the outcome weighting approach in some settings. In the paper by Blanche et al. (2023), a simulation setting reveals considerable bias of the variance estimator in the individual weighting approach. The bias of the variance estimator in the pseudo-observation approach was studied by Overgaard et al. (2018) where the bias was found to be large in extreme cases, but tolerable in many less extreme cases. This seems to be in line with the simulation results of this paper. Generally, it should be possible to construct a more asymptotically appropriate variance estimate by emulating the asymptotic variance expression empirically. Suggestions along these lines are given in Appendix A.2 of Blanche et al. (2023) for the individual and outcome weighting approaches and in the end of Sect. 3 of Overgaard et al. (2017) for the pseudo-observation approach. These variance estimators have not been studied in this paper.

The three approaches studied in this paper are rather simple and should not generally be expected to be efficient. A possible refinement is to consider an actual model of the censoring distribution and using fits from such a model in the weight. This is in fact considered by for instance (Robins and Rotnitzky 1992) for the individual weighting approach, by Scheike et al. (2008) for the outcome weighting approach, and by Binder et al. (2014) for the pseudo-observation approach. By better fitting the censoring distribution, a reduction in asymptotic variance of the regression parameter estimates can be expected. Such approaches may also be augmented by including additional terms in the estimating equation that are designed to reduce the asymptotic variance. Including such terms tends to require a working model for the event time and type as well. Such an approach is also considered by Blanche et al. (2023), citing Robins and Rotnitzky (1992) and Bang and Tsiatis (2000). It is interesting how the augmented approach by Blanche et al. (2023) results in the same asymptotic variance for both types as stated in their Proposition 2. On closer inspection of the augmentation term it can be seen that it achieves reducing \(\Phi _{type }(s \mathbin {\mid }Z)\) to \(\Phi (s \mathbin {\mid }Z)\), in the notation of Observation 1, for both types. As discussed by Blanche et al. (2023), it may in practice be difficult to estimate the augmentation term precisely and this task is further complicated if modeling choices are required to be prespecified. Another refined approach is considered in Martinussen and Scheike (2023) where the observed data efficient influence function is emulated directly in a setting concerning risk regression. Recently, an approach based on so-called censoring unbiased transformations was suggested by Sandqvist (2024), and this approach can in this context best be seen as a refinement of the pseudo-observation approach using working models of the censoring and the outcome to calculate an improved pseudo-observation while obtaining double robustness and oracle efficiency properties. It has been beyond the scope of this paper to study these approaches more closely. In contrast, the focus has here been on simple methods that can be applied with access only to standard tools for statistical analysis, such as tools for the Kaplan–Meier estimator, estimation of parameters in a generalized linear model, and the standard sandwich variance estimator. The assumptions considered do not seem overly restrictive. The most restrictive assumptions are likely the ones imposed on the censoring mechanism. These assumptions can be made less restrictive by appealing to stratification in the censoring distribution estimation. It would be of interest to study whether a considerable amount of efficiency can in fact be gained by applying the more advanced approaches mentioned above, or if it is possible in many settings to achieve a reasonable amount of efficiency by applying an appropriate amount of stratification.