Generation and validation of a surrogate truth model for xSPECT Bone concordance analysis

Objectives We previously reported a high concordance and increased confidence when reading bone scans reconstructed with xSPECT Bone (xB) compared to conventional Flash 3D (F3D;OSEM3D) [SNMMI 2014]. From the same data set, we constructed and validated a surrogate truth model for 228 lesions from 76 patients for future ROC analysis.

Methods Following the concordance reading (76 patients from breast or prostate carcinoma, 3 VOI/Pat, 4 modalities, 9 readers; 2052 randomized reads), the truth reading was conducted, where readers reviewed F3D or xB scans, displayed together with CT, whole body planar and clinical reports. In order to establish a surrogate truth for all lesions, we clustered interpretations into two categories: (1) malignant lesion found (2) no malignant lesion found and use majority to arrive at xB Truth and F3D Truth. Remaining discrepancy is resolved by clinical report. This methodology is initially validated in an adjudication meeting by 6 physicians on 5 “difficult” lesions. The result is a final truth model, and validated using correlation and Intraclass Correlation Coefficient(ICC) analysis.

Results Out of total 2052 lesion truth readings, 1943 (95%) were matched to the truth model using xB while 1935 (94%) were matched using F3D. The correlation between readings and model was high with a chi-square 1635 for xB and 1604 for F3D. Concordance analysis showed that between 9 readers xB yielded an ICC of 0.979 (95% confidence interval 0.975-0.983), and F3D yielded an ICC of 0.976 (95% IC 0.972 to 0.981).

Conclusions We generated a surrogate truth model from a concordance study, and validated it against truth readings with high matching rate (>94%) with both xB and F3D. The concordance between 9 readers is equally high with both xB and F3D with about 0.98 ICC.