Inter- and intra-reader reproducibility of 18F-rhPSMA-7.3 PET image interpretation in patients with suspected prostate cancer recurrence: Results from a phase 3, prospective, multicenter study (SPOTLIGHT)

Introduction: Prostate-specific membrane antigen (PSMA) positron emission tomography (PET) has become standard of care for prostate cancer imaging as demonstrated by its inclusion in the most recent National Comprehensive Cancer Network Guidelines. The SPOTLIGHT trial (NCT04186845) evaluated the diagnostic performance of ¹⁸F-rhPSMA-7.3, a novel high affinity PSMA-targeting PET radiopharmaceutical with potential for low bladder activity. Here, we report the inter- and intra-reader reproducibility of ¹⁸F-rhPSMA-7.3 image interpretation.

Methods: Men with suspected biochemical recurrence of prostate cancer underwent PET/computed tomography (CT) 50-70 min after IV administration of 8 mCi (296 MBq) ± 20% ¹⁸F-rhPSMA-7.3. Scans were evaluated by 3 independent central PET readers who had completed ¹⁸F-rhPSMA-7.3 scan interpretation training and were blinded to all clinical information and results of conventional or confirmatory imaging.

The inter-reader agreement was assessed using the Cohen’s kappa (κ) statistic for pairwise agreement between any 2 readers. Overall agreement between all 3 readers was assessed using Fleiss’ κ coefficient. This same calculation was repeated for the following 3 regions: prostate/prostate bed, pelvic lymph nodes, and other (extra-pelvic) sites (lymph nodes outside pelvis, soft tissue/parenchyma, and bones).

Cohen’s κ was also used to assess the intra-reader agreement between the first read and a repeat read (after 4 weeks) of 20% of randomly selected images for each of the three readers.

Results: Inter-reader agreement for image interpretation in 389 men with evaluable ¹⁸F-rhPSMA-7.3 PET/CT are presented in Table 1. Overall agreement was >75% across the pair‑wise inter-reader comparisons with a Fleiss’ κ = 0.38 across all 3 readers. On a regional basis, the agreement was >80% across the pair‑wise inter-reader comparisons for all regions except the prostate/prostate bed, with a Cohen’s κ ≥ 0.63 for pelvic lymph nodes and other (extra-pelvic) regions. A Fleiss’ κ ≥ 0.66 was achieved for the overall comparison across all 3 readers in these regions. For the prostate/prostate bed, pairwise agreement ranged from 59% to 84% and a Fleiss’ κ of 0.40 was achieved for the overall agreement across the 3 readers.

The intra-reader agreement (Table 2) was > 85% for each reader, with Cohen’s κ ranging from 0.46 to 0.73 across the 3 readers. On a regional basis, intra-reader agreement was broadly high for each reader, with Cohen’s κ shown to be higher for pelvic lymph nodes and other (extra-pelvic) regions than for the prostate/prostate bed.

Conclusions: These data suggest a high degree of inter- and intra-reader agreement across 3 blinded readers given the same set of scans and having completed identical training. Inter-reader agreement was > 75% overall and greatest for the pelvic lymph node region, with > 87% concordance. Intra-reader agreement was > 85% overall.

A low Cohen’s κ for corresponding high agreement data was observed in the inter-reader analysis. This is consistent with the well-reported high agreement, low κ paradox that is a consequence of the κ value’s propensity to be affected by the prevalence of rare findings.

While reproducibility was lower for the prostate/prostate bed than other regions, the substantial reproducibility for extra-prostate regions is of clinical importance due to the potential of such findings to influence treatment selection.

• Download figure
• Open in new tab
• Download powerpoint