Quantifying rod visibility in standardized SPECT quality assurance images

Objectives: Quality assurance of SPECT phantom images often includes a visual assessment of the smallest size of rods that are considered to be adequately resolved. However quantitative scoring is preferred, to avoid intra- and inter-observer disagreement. This study investigated several candidate quantitative methods and evaluated agreement with visual assessments by expert readers.

Methods: In 50 separate phantom acquisitions, two experienced nuclear medicine physicists independently scored the visibility of cold rod arrays of 6 diameters. Visual dichotomous scores were compared to four quantitative metrics of cold rod array visibility derived from SPECT volumetric data: template-based analysis of (1) Area under ROC curve (“Template-AUC”) and (2) signal-to-noise ratio (“Template-SNR”) from binary classification of rod and background measurements, template-based analysis of (3) rod-to-background Contrast, and texture analysis of (4) normalized gray-level co-occurrence matrix (GLCM) energy (“Energy%”). The Template-AUC and Template-SNR metrics designate rod and background measurements as signal-present and signal-absent, then compute AUC and SNR from the resultant ROC curve as measures of cold rod visibility. The Energy% texture analysis metric, which is a measure of orderliness in an image, is computed as the sum of squared GLCM elements normalized to the maximum result of the six cold rod sectors.

Results: Of 300 dichotomous scores, 166 and 135 were assigned as positive by Observers #1 and #2, respectively, with an inter-rater reliability of κ=0.78 (Cohen’s kappa coefficient). Quantitative metric accuracy was assessed by characterizing the ROC with respect to each observer’s dichotomous readings. Accuracy of all quantitative metrics was very high (see Table), with AUC exceeding 96% in all comparisons. For Observer #1, Template-AUC and Template-SNR were equally accurate and were significantly more accurate than Contrast (p=0.02) or Energy% (p=0.02). For Observer #2, Energy% was similarly accurate to Template-AUC and Template-SNR (p=0.58) and significantly more accurate than Contrast (p=0.0003). Continuous values of Energy% were highly correlated (r = 0.90, p < 0.001) with continuous Template-AUC values.

Conclusions: Both template-based (AUC and SNR) and GLCM Energy% metrics are highly accurate in reproducing the visual impressions of medical physicists as to the visibility of cold rods in standardized SPECT phantom studies. Automated quantitative metrics offer the potential for consistent and accurate scoring for routine SPECT quality assurance.