Visual Abstract
Abstract
Prostate-specific membrane antigen (PSMA) PET has a higher accuracy than CT and bone scans to stage patients with prostate cancer. We do not understand how to apply clinical trial data based on conventional imaging to patients staged using PSMA PET. Therefore, we aimed to evaluate the ability of bone scans to detect osseous metastases using PSMA PET as a reference standard. Methods: In this multicenter retrospective diagnostic study, 167 patients with prostate cancer, who were imaged with bone scans and PSMA PET performed within 100 d, were included for analysis. Each study was interpreted by 3 masked readers, and the results of the PSMA PET were used as the reference standard. Endpoints were positive predictive value (PPV), negative predictive value (NPV), and specificity for bone scans. Additionally, interreader reproducibility, positivity rate, uptake on PSMA PET, and the number of lesions were evaluated. Results: In total, 167 patients were included, with 77 at initial staging, 60 in the biochemical recurrence and castration-sensitive prostate cancer setting, and 30 in the castration-resistant prostate cancer setting. In all patients, the PPV, NPV, and specificity for bone scans were 0.73 (95% CI, 0.61–0.82), 0.82 (95% CI, 0.74–0.88), and 0.82 (95% CI, 0.74–0.88), respectively. In patients at initial staging, the PPV, NPV, and specificity for bone scans were 0.43 (95% CI, 0.26–0.63), 0.94 (95% CI, 0.85–0.98), and 0.80 (95% CI, 0.68–0.88), respectively. Interreader agreement for bone disease was moderate for bone scans (Fleiss κ, 0.51) and substantial for the PSMA PET reference standard (Fleiss κ, 0.80). Conclusion: In this multicenter retrospective study, the PPV of bone scans was low in patients at initial staging, with 57% of positive bone scans being false positives. This suggests that a large proportion of patients considered low-volume metastatic by the bone scan actually had localized disease, which is critical when applying clinical data from trials such as the STAMPEDE M1 radiation therapy trial to patients being staged with PSMA PET.
Prostate-specific membrane antigen (PSMA) PET has become the standard imaging modality for patients with prostate cancer at initial staging and biochemical recurrence (1). Previous work has compared the results of PSMA PET with the results of bone scans for the detection of osseous metastases and has shown that PSMA PET has a sensitivity and specificity higher than that of bone scans (2,3). Nearly all prostate cancer trials have used conventional imaging (bone scans in combination with CT scans) for staging. However, how to apply the data from these clinical trials to patients staged with PSMA PET remains unclear.
In the CHAARTED trial, patients with high-volume disease, which is defined as having 4 or more bone lesions on a bone scan, were shown to have overall survival benefits with the addition of docetaxel compared with androgen-deprivation therapy alone (4). Given that PSMA PET has a detection sensitivity higher than that of bone scans, most believe that a larger number of lesions seen on PSMA PET are needed to meet the definition of high-volume disease.
More recently, a secondary analysis of the STAMPEDE M1 radiation therapy (RT) trial demonstrated that patients with low-volume disease, which is defined as having 3 or fewer lesions on a bone scan, benefit from prostate bed RT when combined with androgen-deprivation therapy and docetaxel (5). The results of this study have led to the use of prostate bed RT in patients with low-volume metastatic disease on bone scans. However, questions remain about the accuracy of bone scans for the assessment of metastatic burden and what number of lesions seen on PSMA PET would cause patients to be considered to have high-volume disease and thus not to benefit from prostate bed RT.
We therefore set forth to understand the positive predictive value (PPV), negative predictive value (NPV), and specificity of bone scans using PSMA PET as a reference standard in patients at various disease stages to help us understand how to use PSMA PET results in the setting of clinical trial data performed using conventional imaging.
MATERIALS AND METHODS
This was an international multicenter retrospective head-to-head comparison imaging study. Databases from 3 institutions (University of California, San Francisco, UCLA, and Essen) were retrospectively screened, and patients were included who had PSMA PET scans and bone scans performed within 100 d of one another. Patients with new interval treatment were excluded. Patients included in the analysis were those at initial staging, in a biochemical recurrence and castration-sensitive prostate cancer (BCR/CSPC) setting, and in a castration-resistant prostate cancer (CRPC) setting. This study was approved by each site’s institutional review board; all data were deidentified, and informed consent was waived.
Image Interpretation
Each bone scan was interpreted by 3 masked readers, and each PSMA PET scan was interpreted by 3 other masked readers. Anonymized datasets were provided to each reader. Readers were masked to all clinical information and did not have access to other imaging studies available for the patient. For bone scans, the presence of prostate cancer (positive vs. negative) was recorded for 17 osseous regions. For PSMA PET scans, in addition to the osseous regions used for bone scans, 4 other regions were recorded including prostate bed, pelvic nodes, extrapelvic nodes, and visceral metastases. SUVmax was recorded for the osseous lesion with the highest uptake. The total number of osseous lesions was also noted by each reader for both bone scans and PSMA PET scans. Their findings were entered by the readers directly into the central REDCap database (supplemental material, available at http://jnm.snmjournals.org). Per-region majority rule was used for the analysis.
Statistical Analysis
The primary outcome was PPV, NPV, and specificity of bone scans using PSMA PET as a reference standard. The secondary outcomes were the comparison of patient-level positivity rates, the number of lesions detected with each modality, the interreader agreement, and the PSMA PET SUVmax. Results were broken down by clinical stage. Interreader agreement was evaluated using Fleiss κ and interpreted by criteria of Landis and Koch by region (6). CIs were calculated using the Wilson method (7). A 95% CI was calculated for the number of lesions visualized. A 2-sided Student t test was used to compare SUVmax and the number of lesions seen between cohorts. Statistical analyses were performed with R, version 3.5.1 (R Foundation).
RESULTS
In total, 167 patients were included across 3 institutions (Supplemental Fig. 1). Seventy-seven patients were imaged at initial staging, 60 in the BCR/CSPC setting, and 30 in the CRPC setting (Table 1). The median time between the bone scan and the PSMA PET scan was 29 d (range, 1–100 d). PSMA PET was performed earlier than bone scanning in 117 patients (70%).
Patient Demographics
Imaging Results
On the basis of the majority read, 66 (40%) patients had osseous region disease on bone scans and PSMA PET scans (Table 2). Of the 66 patients positive on bone scanning, 48 were positive on PSMA PET. The PPV and NPV for bone scans were 0.73 (95% CI, 0.61–0.82) and 0.82 (95% CI, 0.74–0.88), respectively, with a specificity of 0.82 (95% CI, 0.74–0.88).
Imaging Results
When focusing on patients at the initial staging, we found that 13 (17%) patients had osseous region disease on PSMA PET, whereas 23 (30%) were identified as having osseous region disease on bone scanning. Of the 23 patients positive on bone scanning, only 10 were positive on PSMA PET (Fig. 1). The PPV and NPV for bone scans were 0.43 (95% CI, 0.26–0.63) and 0.94 (95% CI, 0.85–0.98), respectively, with a specificity of 0.80 (95% CI, 0.68–0.88). When we limited the data to patients who had imaging done within 30 d, the PPV and NPV for bone scans were 0.36 (95% CI, 0.16–0.61) and 0.90 (95% CI, 0.75–0.97), respectively, with a specificity of 0.76 (95% CI, 0.60–0.87). In the 16 patients with fewer than 4 lesions on bone scanning (i.e., low volume by CHAARTED criteria), only 7 (44%) were positive on PSMA PET. Three patients were positive on PSMA PET and negative on bone scanning.
A 74-y-old man at initial staging with PSMA PET that demonstrates low uptake in primary tumor (A, arrowhead) and left pelvic side wall node (A, arrow). Bone scan was read as positive by 2 of 3 readers. One reader read 2 lesions in lumbar spine (C, arrows), and second reader read additional lesions in ribs, thoracic spine, and sacrum (C, arrowheads). This case is false-positive on bone scan.
In the BCR/CSPC setting, the PPV, NPV, and specificity were 0.77 (95% CI, 0.57–0.90), 0.74 (95% CI, 0.58–0.85), and 0.85 (95% CI, 0.69–0.93), respectively. In CRPC patients, the PPV, NPV, and specificity were 1.0 (95% CI, 0.85–1.0), 0.56 (95% CI, 0.27–0.81), and 1.0 (95% CI, 0.57–1.0), respectively. Of the 16 patients in the BCR/CSPC and CRPC settings with fewer than 4 lesions on bone scanning, 12 (75%) were positive on PSMA PET.
In patients with positive osseous lesions on both PSMA PET and bone scanning by a majority read (n = 48), there were 4.4 ± 0.6 lesions on bone scanning and 7.5 ± 1.6 lesions on PSMA PET (P < 0.001). In the subset of patients at initial staging, the number of lesions seen was smaller and the difference not significant, with 2.8 ± 1.1 lesions on bone scanning and 3.2 ± 1.3 lesions on PSMA PET (P = 0.5). The mean SUVmax on PSMA PET in patients at initial staging was lower than that in patients in the BCR/CSPC and CRPC settings (10.3 ± 6.9 vs. 36.3 ± 42.5, P = 0.049).
Interreader Agreement
For osseous lesions, interreader agreement for bone scans was moderate (Fleiss κ, 0.51), whereas for PSMA PET, it was substantial (Fleiss κ, 0.80). For prostate bed, pelvic nodes, and extrapelvic nodes, the interreader agreement for PSMA PET was substantial: 0.73, 0.61, and 0.73, respectively. For other organs, the interreader agreement for PSMA PET was moderate (0.41).
DISCUSSION
In this multicenter retrospective head-to-head comparison diagnostic study using PSMA PET as a reference standard, we demonstrated that the specificity of bone scans is high and similar across disease states, but the PPV is lower in patients at initial staging than in the population as a whole (0.43 vs. 0.73). Overall, 27% of patients with osseous metastases on bone scanning were negative on PSMA PET, and this increased to 57% in patients at initial staging.
Prior work has demonstrated similar high specificities for bone scans, reporting values of approximately 0.82 (8). It is important to remember that in the initial staging population, where the prevalence of osseous metastases is lower, the false-positive rate is much higher. This can be seen in our data, in which the percentage of bone scans that were false positive was 57% at initial staging versus 0% in the CRPC setting. These results are nearly identical to prior work comparing the results of bone scanning with PSMA PET at initial staging (9).
These results bring into question how to define patients with low-volume disease using PSMA PET in light of the STAMPEDE M1 RT data. If one were to apply our initial staging data to the M1 RT trial, 56% of patients with low-volume disease based on bone scanning had localized disease by PSMA PET. Therefore, there is a greater likelihood that the overall survival benefit seen in the trial is not driven by preventing further development of metastatic disease but rather by providing definitive RT to nonmetastatic disease that was incorrectly classified as M1 by bone scanning. Although one typically thinks of bone scans as understaging disease relative to PSMA PET (as was shown in our study, because more lesions than on bone scans were seen in patients with positive PSMA PET scans), in the initial staging setting a bone scan overstages patients relative to PSMA PET because of its high false-positive rate.
This study has many limitations. First, using PSMA as a reference standard for evaluation of bone scans can be problematic, particularly given the false positives on PSMA PET in bone lesions. Of note, PSMA PET false positives would not impact the false-negative rate of bone scans in our analysis. Second, the bone scans and PSMA PET scans were not concurrent, although the difference in PPV was not impacted by limiting imaging studies done 30 d apart versus 100 d apart. Third, planar bone scans were used consistently with the STAMPEDE trial, but it is well established that the use of SPECT/CT would increase the specificity and PPV of bone scans (10). It should be noted that the STAMPEDE trial did not require SPECT/CT. Fourth, as the masked reads were performed retrospectively, readers may have been inclined to increase their sensitivity.
CONCLUSION
In this multicenter retrospective diagnostic study using PSMA PET as the reference standard, the PPV of bone scans at initial staging was low (0.43). This results in incorrect staging (as having osseous metastasis) of more than half of patients in this group. This overstaging by bone scans is important when applying clinical data from trials such as the STAMPEDE M1 RT trial to patients being staged with PSMA PET. Before patients receive prostate bed RT in low-volume metastatic disease on PSMA PET, further work should be performed to understand if the results of the STAMPEDE M1 RT trial are generalizable.
DISCLOSURE
Thomas Hope has grant funding to the institution from Clovis Oncology, Philips, GE Healthcare, Lantheus, the Prostate Cancer Foundation, and the National Cancer Institute (R01CA235741 and R01CA212148); received personal fees from Ipsen, Bayer, and BlueEarth Diagnostics; and received fees from and has an equity interest in RayzeBio and Curium. Wolfgang Fendler reports fees from SOFIE Biosciences (research funding), Janssen (consultant, speaker), Calyx (consultant), Bayer (consultant, speaker, research funding), Parexel (image review), Novartis (speaker), and Telix (speaker) outside the submitted work. No other potential conflict of interest relevant to this article was reported.
KEY POINTS
QUESTION: What is the PPV, NPV, and specificity of bone scans in prostate cancer patients using PSMA PET as a reference standard?
PERTINENT FINDINGS: Although the specificity of bone scans was similar in initial staging of patients versus the entire cohort (0.80 vs. 0.82), the PPV was markedly lower in the initial staging of patients (0.43 vs. 0.73).
IMPLICATIONS FOR PATIENT CARE: The lower PPV at the initial staging of patients means that more than half of patients staged as having metastatic disease on bone scans actually had no osseous metastasis. The observed overstaging by bone scans is important when applying clinical data from trials such as the STAMPEDE M1 RT trial to patients being staged with PSMA PET.
Footnotes
Guest Editor: Michael Hofman, Peter MacCallum Cancer Centre
Published online Aug. 17, 2023.
- © 2023 by the Society of Nuclear Medicine and Molecular Imaging.
REFERENCES
- Received for publication April 21, 2023.
- Revision received July 5, 2023.
In this issue
Jump to section
Related Articles
Cited By...
- Low- and High-Volume Disease in Metastatic Hormone-Sensitive Prostate Cancer: From CHAARTED to PSMA PET--An International Multicenter Retrospective Study
- FAP and PSMA Expression by Immunohistochemistry and PET Imaging in Castration-Resistant Prostate Cancer: A Translational Pilot Study
- [68Ga]Ga-PSMA-11 PET and Prostate Cancer Bone Metastases: Diagnostic Performance of Available Standardized Criteria
- Out with the Old, in with the New: Can We Bridge the Gap Between Clinical Trial Results Based on Bone Scans and the Era of Modern Prostate Cancer Imaging?