Prediction vs. Prognostication and Guarantee-Time Bias: Steering Clear of the Pitfalls of Interpreting Observational Data

TO THE EDITOR: Some of the findings reported recently by Etchebehere et al. on the use of ²²³Ra in patients with metastatic castrate-resistant prostate cancer and the utility of baseline ¹⁸F-fluoride PET/CT to predict outcome merit some critical perspective (1).

First, the authors state that the “skeletal tumor burden determined by fluoride PET/CT will be correlated with clinical outcomes” in order to “identify patients who will not respond to ²²³Ra, thus reducing morbidity and unnecessary costs.” This description clearly fits the aim of validating a predictive biomarker for a specific therapy. However, this question cannot reasonably be answered using the single-arm cohort used by the authors. Instead a two-arm trial would have been required, where one group received the treatment and the other group did not, allowing for a formal test of the interaction between treatment and biomarker status. Only then can it be shown that the biomarker is indeed specific for the therapy and not just for the disease condition. Unfortunately, the authors cause further confusion by stating that the purpose of the study was to evaluate “the potential of fluoride PET/CT” as “a prognostic biomarker of survival.” Prognostication refers to outcome independent of therapy, and this can indeed be assessed using a single-cohort design. Excellent perspectives on the differences between predictive and prognostic biomarkers have been published elsewhere (2,3). The conclusion of the study should therefore have read: skeletal tumor burden on baseline ¹⁸F-fluoride PET/CT is a prognostic biomarker of overall survival in patients treated with ²²³Ra.

Second, the authors analyzed survival outcomes as a function of the number of ²²³Ra treatment cycles received, finding that “the number of ²²³Ra doses was an independent predictor of [overall survival], was beneficial, and reduced the risk of death” and a similar conclusion for progression-free survival. However, this analysis should not have been performed, as it cannot be justified in a single-arm study without a control group. In particular, it is jeopardized by guarantee-time bias. This bias occurs when analyzing outcomes based on a categorizing variable that is measured at a different time-point for every individual under study (in this case the total number of treatment cycles) relative to the individual’s time of origin (e.g., start of treatment). Patients in whom the categorizing event occurs later are almost guaranteed (hence the name) to have a longer survival or time to the event of interest, regardless of the intervention (4).

To elaborate, the total number of cycles received is a time-varying covariate, as being able to receive an additional treatment cycle is conditional on having survived (or the disease not having progressed) until that time point. It is thus not only determined by the effectiveness of the therapeutic agent but also by tumor aggressiveness, prior therapies, patient condition, and many other confounding factors. It should be noted that a multivariable time-to-event analysis is unable to adjust for this bias unless the counting process is modified, observations are weighted, or the analysis clock is reset at a specified landmark (4). The statistical methods section of the paper does not provide evidence for this. Though the authors do note that “although it seems logical that [progression-free survival] would be longer in patients who received more ²²³Ra doses in comparison to those who did not receive doses because of death, this was not the case,” they do not recognize the greater danger of guarantee-time bias on the validity of their results. Put differently, if this type of analysis would be appropriate, the randomized controlled trial would no longer be required to determine treatment efficacy.

To illustrate this point further, I have used a publically available clinical trial dataset released through Project Data Sphere that includes individual patient data of the comparator arm (n = 285) of a phase III randomized controlled trial of sunitinib compared with placebo in metastatic castrate-resistant prostate cancer patients who had progressed after docetaxel chemotherapy (5,6). This population is likely to be similar to that enrolled in the phase III trial of ²²³Ra (ALSYMPCA) (7). For every completed 28-d treatment cycle of placebo, there was a highly statistically significant apparent reduction of 21% in the hazard of death (unadjusted hazard ratio, 0.79; 95% confidence interval, 0.74–0.84; P < 0.0001) and of 38% in the hazard of the disease progressing or death (unadjusted hazard ratio, 0.62; 95% confidence interval, 0.57–0.66; P < 0.0001). When one accepts the randomized controlled trial paradigm, which is at the heart of contemporary drug evaluation, no therapeutic effect can be attributed to placebo and these findings can therefore be explained only by bias, including guarantee-time bias.

In conclusion, ¹⁸F-fluoride PET/CT appears to be a prognostic imaging biomarker in the cohort of men with metastatic castrate-resistant prostate cancer receiving ²²³Ra. Whether it is really predictive of ²²³Ra benefit remains to be determined, and it would be premature and unwarranted to guide treatment decisions with respect to ²²³Ra on baseline ¹⁸F-fluoride PET/CT findings at this time.

• © 2016 by the Society of Nuclear Medicine and Molecular Imaging, Inc.