Application of Lesion Detectability Phantoms for Model Observer Assessment in PET Imaging

Objectives: It remains challenging to quantitatively evaluate and compare detectability performance on PET systems. The goals of this work were to design a fillable PET phantom with lumpy background and low-contrast detectability features as well as demonstrate its use in comparing scanners and algorithms.

Methods: 3-D printed, 12-sided nylon regular dodecahedrons were used with 0.75 mm diameter elements. These created 3:1 contrast features when placed into a fillable phantom in a background of 4 mm acrylic beads. A pre-mixed FDG solution was poured into each a cylindrical head-size and elliptical body-size phantom that included between 42 and 83 small contrast features, at one or more sizes between 4 and 7 mm, placed in a regular pattern within the imaging volume. Feature placement in the phantoms was such that many signal-present and signal-absent sub-images could be extracted from scan (and sub-scan) data and used as input for model observers, in our case a channelized Hotelling observer (CHO) with 3 difference-of-Gaussian channel model including internal noise. A head-size phantom was scanned on three PET systems at equal decays per scan and a body-size phantom was scanned on one system. Use of multiple scans per scanner enabled comparison at different count levels and with different reconstruction methods. Scanning was performed at clinically relevant activity concentrations (2.5-10 kBq/mL) and scan durations. Observer area under the ROC curve (AUC) results were generated including a 95% confidence interval.

Results: Using the head-size phantom, the AUC increased as the PET system sensitivity increased, as expected. Higher AUC was found when including PSF, TOF and fully-convergent regularized iterative reconstruction - on average, AUC increase was 1.8%, 6.3% and 1.6%, respectively. Both head-size and body-size phantoms demonstrated algorithm and system design differences, with inclusion of TOF information having more impact on the body-size phantom. The AUC variability also increased using the body-size phantom, from approximately 4.4% on average with the head-size phantom to 6.6% with the body-size phantom. Detectability performance at multiple count levels, via un-listing list-mode data at more sub-scans per scan, demonstrated similar differences between systems and algorithms.

Conclusion: The phantom design and scanning methodology has demonstrated the capability to quantitatively compare model observer detectability between multiple PET systems and image reconstruction methods. The methods can be adapted to assess performance at different count-rate conditions, phantom geometries and potentially at other feature contrasts. This approach could be used in conjunction with conventional quantitation phantoms to improve acquisition and image generation techniques.