Human-Observer Receiver-Operating-Characteristic Evaluation of Attenuation, Scatter, and Resolution Compensation Strategies for ^99mTc Myocardial Perfusion Imaging

Patients

Image sets from 100 patients who underwent ^99mTc-sestamibi stress studies were used in this ROC investigation. Fifty-five of these patients (11 women and 44 men) underwent cardiac catheterization. A patient was classified as not healthy when at least 1 of the 3 major coronary artery territories had a 50% or greater narrowing in luminal diameter, as determined by angiography. Of the 55 patients who underwent catheterization, 8 were classified as healthy, 15 had single-vessel disease, 19 had double-vessel disease, and 13 had triple-vessel disease. The distribution of the abnormal territories was as follows: 29 abnormal LAD, 28 abnormal left circumflex coronary (LCx), and 35 abnormal RCA territories. The remaining 45 patients (21 women and 24 men) with <5% likelihood for CAD (9) were classified as healthy. All patients underwent either physical stress testing according to the Bruce protocol or pharmacologic stress (dipyridamole) testing. Informed consent was obtained from all patients for use of their data.

Data Acquisition

Clinical stress studies acquired on our PRISM 3000XP 3-headed SPECT system (Philips Medical Systems) were used in these ROC investigations. For the simultaneous acquisition protocol used, 2 low-energy, high-resolution parallel-hole collimators acquired ^99mTc emission projections, and head 3, fitted with a 65-cm focal-length fanbeam collimator opposed to a ¹⁵³Gd line source, acquired transmission data. The activity of the ¹⁵³Gd line source varied between 8,880 MBq (240 mCi) and 3,108 MBq (84 mCi) during the year of use. Three interchangeable tungsten filters varying in thickness from 0.10 to 0.025 cm were used to compensate approximately for the radioactive decay of the line source, thereby decreasing the variation in source intensity during the year of use.

Three energy windows were used to acquire emission data on heads 1 and 2, which had the parallel-hole collimators. These windows were a ^99mTc photopeak window centered at 140 keV with a 15% width, a large ^99mTc Compton scatter window centered at 106 keV with a 25% width for determining the body outline for use in constraining the fanbeam transmission reconstructions (10,11), and a window centered at 123 keV with a 4% width for performing triple-energy-window (TEW) SC (12) of the ^99mTc photopeak data. Head 3, which had the fanbeam collimator, acquired 4 energy windows, 3 for transmission imaging and 1 for emission imaging. The transmission windows consisted of a ¹⁵³Gd photopeak window centered at 100.8 keV with a 15% width and 2 scatter windows centered at 123 keV with a 4% width and 84 keV with a 6% width for performing TEW correction (13) of the ¹⁵³Gd transmission photopeak data. The emission window was a ^99mTc photopeak window centered at 140 keV with a 15% width. Data from this window were not used for the research reported in the article. All projections were acquired on a 64 × 64 matrix with a pixel size of 0.634 cm resulting from the use of a magnification of 1.123. Acquisition was done for 60 angles covering a circular 360° acquisition orbit in a step-and-shoot mode with an acquisition time of 16 s per frame.

Image Reconstruction

The transmission projections were corrected for resolving-time errors. Resolving-time correction was done with a paralyzable model and a resolving time measured by use of the paired-source method of Adams et al. (14). The total number of counts per second for the acquired frame was input into the model as the observed-frame counting rate, and the true-frame counting rate was solved for by use of the Newton-Raphson method (15). Each pixel count in the frame was corrected for count loss by multiplication by the ratio of the estimated true frame-counting rate to the observed frame-counting rate. The ^99mTc cross talk in the ¹⁵³Gd transmission projection data was estimated by use of the TEW method of Ogawa et al. (12). The attenuation maps were reconstructed by use of 30 iterations of the maximum-likelihood transmission gradient algorithm with a gamma prior. The filtered TEW down-scatter estimate (Butterworth filter of order 4 and a cutoff frequency of 0.15 cycle per second) was explicitly modeled in the reconstruction algorithm (13), avoiding a preliminary scatter subtraction step. To reduce the effects of truncation, the patient body outline (estimated from Compton-scattered ^99mTc projection data) was used as prior support, reconstructing only pixels confined within the outline (10,11). Finally, the reconstructed ¹⁵³Gd attenuation maps were scaled from ¹⁵³Gd to ^99mTc before emission reconstruction.

Four different reconstruction strategies were considered for evaluation: filtered backprojection (FBP), ordered-subset expectation maximization (OSEM) (16) with AC alone, OSEM with AC and SC (AC + SC), and OSEM with AC, SC, and RC (AC + SC + RC). FBP reconstructions of the ^99mTc emission images were implemented with a ramp filter after the projections were prefiltered with a 2-dimensional (2D) Butterworth filter of order 5 and a cutoff frequency of 0.25 cycle per pixel, as used clinically (17). Reconstruction was done over 180° from right anterior oblique to left posterior oblique.

For OSEM reconstructions, we used 15 subsets (4 angles per subset) and corrected the ^99mTc projections for resolving-time errors and radioisotope decay before the 360° reconstruction was done. The number of iterations of OSEM and the SD for the postreconstruction 3-dimensional (3D) gaussian filter applied to the slices were selected as those that resulted in the largest area under the ROC curve (A_z) in human-observer ROC studies conducted with hybrid images (18,19). Reconstructions with AC were implemented by use of 1 iteration of OSEM followed by application of a postreconstruction 3D gaussian filter with an SD of 0.75 pixel. The scatter component was estimated by use of the TEW method (12) and prefiltered with a Butterworth filter of order 4 and a cutoff frequency of 0.2 cycle per pixel (20). Reconstructions with AC + SC were implemented by use of 1 iteration of OSEM followed by application of a postreconstruction 3D gaussian filter with an SD of 0.75 pixel. As noted earlier for the transmission reconstructions, the scatter component was again included in the projection-backprojection step of iterative reconstructions (20,21).

Finally, the emission images were reconstructed with all 3 corrections (i.e., AC, SC, and RC) by use of 5 iterations of OSEM followed by application of a postreconstruction 3D gaussian filter with an SD of 0.75 pixel. The importance of accurate 3D combined correction for these degradations on the basis of image-quality metrics was demonstrated previously (22–24). In this study, DDR was modeled in the projection-backprojection step with gaussian diffusion (25,26). This strategy provides an efficient method for incorporating a model of the system spatial resolution as a distance-dependent 2D gaussian function. In this study, the full width at half maximum in the axial and transaxial directions for the PRISM 3000XP with low-energy, high-resolution collimators was measured as a function of distance from the acquisitions of point sources. These measured values were fitted by linear regression and used to model the change in full width at half maximum as a function of distance from the collimator face. Such modeling has been shown to improve 3D spatial resolution (27) and to provide improved quality of simulated cardiac images, as judged by image-based metrics (28,29). It has been shown that this form of RC does degrade cardiac wall count uniformity compared with the results obtained when AC alone is used, unless heavy postreconstruction 3D gaussian filtering is applied, thus raising a question as to its clinical utility for cardiac imaging (29). Gifford et al. previously showed that RC with gaussian diffusion results in an improvement in lesion detection accuracy in human-observer localization-ROC studies with simulated ⁶⁷Ga images (30); however, the ability of this technique to improve the detection of perfusion defects in cardiac imaging has not been investigated. Thus, in this study we compared OSEM with AC + SC to OSEM with AC + SC + RC to determine whether this form of 3D compensation for DDR increases CAD detection accuracy on clinical images.

ROC Comparison

We used a total of 7 observers in this ROC investigation. Three were attending nuclear cardiologists, and the remaining 4 were cardiology fellows. This grouping of observers enabled the evaluation of any differences in observer performance that might arise because of differences in the extent of observer experience.

Because inadequate training can have a significant impact on observer performance (1), this investigation was designed to provide the observers with training in each of the reconstruction strategies before the actual ROC investigation. It is important to note that the patient images used in all of the training sessions were distinct from the 100 images used in the actual evaluation. The additional patient studies were selected from our database of clinical studies, from which those used for the ROC investigation were drawn. However, in the training studies, neither catheterization nor fulfillment of the criteria necessary for the studies to have a low likelihood for CAD was available, because all such studies were saved for the ROC investigation. Thus, the “truth” with regard to the presence or absence of CAD for the training studies was taken to be the recorded clinical interpretation based on criteria such as the evaluation of rest and stress FBP perfusion images, raw projection data, polar maps, wall motion assessment, and clinical presentation at the time of imaging. Only studies that were clearly abnormal or normal were used for training, and the observer scores were used only by the observers themselves in developing their criteria for judging confidence regarding the presence of CAD.

Training was conducted in 3 phases. In the first phase, the investigators provided the observers with the review article of Corbett and Ficaro (1) and discussed the relevant issues raised in this article regarding variations in the appearance of normal studies with compensation. Next, they reviewed with the observers the average polar maps from normal studies processed by the various strategies (31) and pointed out variations that might be expected from FBP reconstructions. Next, the observers were brought to the ROC reading area and instructed in the operation of the graphical user interface (GUI) to be used (Fig. 1).

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1.

GUI used in ROC studies. Top 3 panels display SA, HLA, and VLA slices, from top to bottom, for interpretation by observer. Three pull-down buttons shown on polar map marking 3 arterial territories (middle panel at bottom) allowed user to assign score to each vascular territory (LAD, LCx, and RCA). During training, feedback regarding presence or absence of defect would appear in right panel at bottom of interface after observer completed assigning scores as described in text. Instructions for observer were given in left panel at bottom.

The second phase of training was conducted to enable the observers to make a direct comparison between FBP and the different iterative methods with compensation strategies. Here, the aim was for the observers to observe differences in the quality of corrected images, such as a lower apparent localization in the apex and a higher apparent subdiaphragmatic concentration, that were suggested to them in a preliminary discussion of possible variations. In each reading session, the observers viewed slices of both FBP reconstructions and one of the iterative OSEM reconstructions with some form of compensation, that is, AC alone, AC + SC, or AC + SC + RC. The observers were unaware of which OSEM reconstruction strategy they were viewing (a masking strategy) but were provided with knowledge of which slices were reconstructed by FBP. The observers viewed a total of 15 images (10 normal and 5 abnormal) for each iterative method (along with the FBP images) in 3 separate reading sessions. Also, feedback was provided with regard to the presence or absence of a perfusion defect in each of the 3 territories, that is, LAD, LCx, and RCA, for each training image set.

In the third phase, the observers viewed images for the 4 reconstruction methods separately in masked sessions. In each session, they viewed a total of 30 images (20 normal and 10 abnormal). Again, feedback was provided with regard to the presence or absence of a perfusion defect in each of the 3 territories. The reconstruction strategies were identified to the observers with a randomly selected Roman numeral between I and IV. The observers were informed that the purposes of this phase of training were to familiarize them with the GUI and to allow them to develop their own criteria for interpreting the strategies, because the same numeral would be used in the forthcoming ROC investigation.

Upon completion of the 3 phases of training, the observers started the reading sessions for the ROC investigation. The 100 patient images in this investigation were split into 2 groups. Because 4 reconstruction strategies were being evaluated in this work, the observers performed a total of 8 reading sessions. In each of these reading sessions, the observers viewed images for a single reconstruction strategy. Each observer evaluated the images independently in masked reading sessions. These reading sessions were performed in a quiet area, away from the clinic. The lighting in the area could be adjusted independently by the observers according to their preferences. The reconstructed transverse slices were magnified and resliced into short-axis (SA), horizontal long-axis (HLA), and vertical long-axis (VLA) views and presented for visual interpretation by use of the GUI illustrated in Figure 1. The top 3 panels displayed the SA, HLA, and VLA views, from top to bottom, for interpretation by the observer. Although the images were presented by use of the default color scale used in the clinic, the observers had the option of changing to other color scales if they so desired. A 100-point continuous scale, as has been recommended for ROC studies (32,33), was used, and the 3 pull-down buttons shown in the middle panel at the bottom of Figure 1 allowed the user to assign a score for the presence or absence of a perfusion defect in each of the 3 vascular territories (LAD, LCx, and RCA).

At the beginning of each reading session, the observers were retrained with a set of 15 images (5 abnormal and 10 normal) from the set used in training. During retraining, feedback regarding the presence or absence of perfusion defects in each of the 3 territories was provided to the observer, as shown in Figure 1. This step was followed immediately by viewing of a set of 50 images that served as the actual study images. No feedback was provided to the observers when they were scoring these images. An observer’s overall score for the investigation, representing observer performance for the detection of CAD, was taken to be the maximum of the scores in all 3 territories. In order to reduce reading-order effects between observers, the order of reading of the studies was designed on the basis of the guidelines proposed by Metz (34). Also, the images to be scored in each reading session were randomized by the GUI to minimize reading-order effects within each group of images.

ROC curves were generated with the ROCKIT program to estimate the binormal parameters (35) that best fitted the collected ratings data. The estimated A_z values for each observer and each test condition were averaged over the two 50-image viewing sets. The fitted ROC curves were averaged over all observers as well as over the 2 reading sessions to obtain a composite ROC curve for each test condition. This step was accomplished by averaging the true-positive fractions at given false-positive fractions. A 2-way ANOVA was used to test for statistically significant differences between the reconstruction strategies and between observers. When there was a significant difference, the Scheffé multiple-comparisons test (36) was used to determine which pairs of observers or test conditions differed significantly from one another.