Accuracy of 3-Dimensional Reconstruction Algorithms for the High-Resolution Research Tomograph

Reconstruction Algorithms

All implementations of the reconstruction algorithms were developed in-house. At the time of writing, commercial implementations exist for only OP-OSEM and ANW-OSEM, as provided by the manufacturer of the scanner.

Phantoms and Studies

For calibration of the various reconstruction techniques, a homogeneous phantom of 20-cm diameter was used and filled with an ¹⁸F-FDG solution of 8 kBq/mL. In addition, this phantom was used with two 5-cm diameter and 20-cm-long inserts and filled with ¹⁸F-FDG solutions of 36 and 3 kBq/mL. The background compartment was filled with a solution of 8 kBq/mL. A 3D anthropomorphic human brain phantom (Hoffman phantom; Data Spectrum) (25) was used to compare realistic activity-concentration distributions in clinical situations. The phantom was filled with an ¹¹C solution of 66 kBq/mL for a decay or linearity study or with an ¹⁸F-FDG solution of 50 kBq/mL to study the effects of NECs on reconstructed activity concentrations for various frame durations. For all phantom studies, an emission list-mode scan of 1 h was performed. List-mode scans of the homogeneous phantom were histogrammed in multiple frames with different durations (i.e., 5, 10, 30, 60, 300, 1,800, and 3,600 s). The list-mode scan of the Hoffman phantom was histogrammed in 300-s frames (¹¹C decay experiment) or in multiple replicates (frames) with different durations (¹⁸F-FDG experiment): 20 frames of 5, 10, 30, and 60 s; 12 frames of 300 s; and 1 frame of 1,800 and 3,600 s. Transmission scans were performed after activity had decayed to background level.

Dynamic human ¹¹C-flumazenil studies were used for clinical evaluation. In 5 healthy volunteers, a 60.5-min list-mode scan was acquired immediately after administration of ¹¹C-flumazenil (363 ± 42 MBq). Each emission scan was histogrammed in 17 time frames with variable frame lengths (1 × 30, 4 × 15, 4 × 60, 2 × 150, 2 × 300, and 4 × 600 s). Before emission scanning and tracer administration, a 6-min transmission scan was performed. During emission scanning, continuous online arterial blood sampling was performed (26). At set times (5, 10, 15, 20, 30, 40, and 60 min after injection), continuous blood sampling was interrupted to collect manual blood samples. These manual sample data were used for calibrating the (online) blood sampler, for measuring plasma or whole-blood ratios, and for determining metabolite fractions (27). The metabolite-corrected arterial plasma time–activity curve was used as an input function for kinetic analysis. A structural T1-weighted MR image was acquired on a 1.5-T scanner (SONATA; Siemens Medical Solutions) used for region-of-interest (ROI) definitions. The medical ethics committee of the VU University medical center had approved the study, and the subjects had given written informed consent.

Reconstructions

All studies were normalized and corrected for scatter, randoms, attenuation, decay, and dead time. Emission scans were reconstructed using the various 3D iterative reconstruction techniques mentioned above. To assess convergence of all algorithms, the 10- and 3,600-s frames of the inserts phantom experiment and the 3,600-s frame of the ¹⁸F-FDG Hoffman phantom experiment were reconstructed with an increasing number of iterations (up to 30) and 16 subsets. The human brain study was reconstructed with different numbers of iterations but with a smaller range, to investigate the effects on dynamic human brain studies. All other experiments were analyzed at the optimal number of iterations as found with the convergence studies (as will be shown later in this article). In addition, all studies were reconstructed using the 3D-FBP algorithm (8), which was used as a reference for iterative reconstructions.

Parametric Analysis of Human Brain Study

Parametric images were generated using a basis-function-method implementation of a plasma-input single-tissue-compartment model (28,29) providing volume of distribution (V_T) and delivery (K₁) images. Furthermore, reference-tissue parametric analysis was performed using receptor parametric mapping with the pons as the reference region providing relative delivery (R₁) and binding potential (BP_ND) images. Receptor parametric mapping is a basis-function-method implementation of the simplified reference-tissue model (30). These parametric methods were applied to assess the effects of different reconstruction strategies on the (overall) accuracy of kinetic analyses of PET data. Parametric images were generated using PPET (31).

Data Analysis

Assessment of Quantitative Accuracy

To study the various 3D iterative reconstruction techniques quantitatively, all images were compared with those using 3D-FBP, except for the images from the homogeneous phantom studies, which were normalized by dividing the reconstructed activity concentration by the true activity concentration. True activity concentrations were derived by measuring three 0.5-mL samples in a calibrated well counter. Coefficient of variation (COV; percentage) was calculated as the ratio of SD of the reconstructed activity concentration over all replicates to the observed average activity concentration. The average activity concentration was calculated as the average over all replicates.

The Hoffman phantom data were normalized to the measured activity concentration per region using the 3D-FBP image with the highest scan statistics. This procedure was followed because the slabs separating the compartments of the Hoffman phantom are about the same size as the scanner resolution and slice thickness, resulting in a variable partial-volume effect. Consequently, shifting the Hoffman phantom axially by half the detector width will result in a different image. Therefore, normalization to true activity concentration was not accurate.

True kinetic parameters were not known for the human brain study. Therefore, data were normalized by dividing them by the data obtained by 3D-FBP reconstructions. Consequently, only the relative difference in activity concentration with respect to those seen in the data obtained with 3D-FBP can be evaluated.

Convergence of Different Algorithms

Figure 1 shows convergence rates of various iterative reconstruction algorithms for the homogeneous phantom with hot and cold inserts at low (10 s) and high (3,600 s) NECs. All reconstruction algorithms had converged using 10, 16, 7, and 16 iterations for ANW-OSEM, OP-OSEM, OSWLS, and SP-OSEM, respectively. At low NEC, ANW-OSEM showed a bias of 29% for the hot and of 370% for the cold inserts. All other iterative reconstruction algorithms showed bias (21%−47%) for cold regions after convergence, but hardly any bias (<2%) for the hot insert. 3D-FBP showed a bias of −24% for the cold and of 7% for the hot inserts. At high NEC, all reconstruction algorithms showed only limited bias after convergence (∼5% for hot, ∼−10% for cold inserts). For cold regions, however, OSWLS hardly showed any bias (<3%) after convergence was reached.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1. 

Homogeneous phantom with inserts. Effects of increasing number of iterations of various reconstruction algorithms on cold insert with low (10 s) and high (3,600 s) NECs and on hot insert with low and high NECs. Note that OP-OSEM and SP-OSEM overlap.

Figure 2 shows bias versus COV for various iterative reconstruction algorithms in the case of the homogeneous phantom with hot and cold inserts at low (10 s) and high (3,600 s) NECs. After convergence (at a minimal or constant bias level), increasing the number of iterations increased COV at high NECs and increased both bias and COV at low NECs.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 2. 

Homogeneous phantom with inserts. Effects of increasing number of iterations of various reconstruction algorithms on bias and COV in background and cold and hot inserts with low (10 s) and high (3,600 s) NECs. Arrows indicate suggested optimal number of iterations for various iterative reconstruction algorithms.

Convergence rates of various iterative reconstruction algorithms for the Hoffman phantom are shown in Supplemental Figure 1 (supplemental materials are available online only at http://jnm.snmjournals.org). Every reconstruction algorithm converged within 16 iterations to an acceptable GM-to-WM contrast ratio (comparable or better than that of 3D-FBP). The numbers of iterations required to obtain a resolution similar to 3D-FBP were equal to those seen in the inserts phantom study, that is, consistent with the numbers of iterations described above. All reconstruction algorithms showed hardly any bias (<1%) in GM and WM after convergence.

Homogeneous Phantom: Varying NECs

Figure 3 shows results obtained from the homogeneous phantom study without inserts. ANW-OSEM showed a large bias in reconstructed activity concentration, amounting to 66%−96% for short frame durations (5–60 s) or low NECs. OSWLS decreased this effect to 2%−19%. SP-OSEM showed bias of 0%−9% for these frames. Similarly, OP-OSEM showed only 0%−8% bias. 3D-FBP showed almost no bias (<2%) for all NEC or frame durations tested (5–3,600 s), but showed high COV (4.4·10²%−1.5·10³%) and SD (32–110 kBq/mL) for short-duration (5–60 s) frames. OSWLS showed the least, almost linear, increase in COV (1.3·10²%−2.2·10²%) with decreasing NECs (shorter frames).

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 3. 

Results of various reconstruction methods for homogeneous phantom: a reconstruction over true activity concentration at different NECs (A), with corresponding COV (B) and SD (C). Note that OP-OSEM and SP-OSEM overlap.

Homogeneous Phantom with Hot and Cold Inserts: Varying NECs

Background activity concentrations showed trends similar to those seen in Figure 3A for all reconstruction algorithms. The hot and cold inserts showed trends similar to those shown later for the Hoffman phantom in Figures 4A and 4B. However, all reconstruction algorithms overestimated the activity concentration of the hot insert by 5% at high NECs (300–3,600 s). 3D-FBP underestimated the activity concentration of the cold insert by 14%−19% at high NECs.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 4. 

True activity concentration for various reconstructions of Hoffman phantom at different NECs (randoms fraction, ∼17.6%) for GM (A), WM (B), and GM-to-WM contrast ratio (C). Note that OP-OSEM and SP-OSEM overlap. Error bars indicate SD.

Hoffman Phantom: Varying NECs by Varying Frame Durations

All reconstruction algorithms showed no bias (<1%) in activity concentration distribution at high frame lengths (1,800 and 3,600 s; NEC > 3.6·10⁸ counts). In Figure 4, results obtained for the brain phantom (¹⁸F-FDG experiment) are shown for all other frame durations (NEC). Performance of all iterative reconstructions depends on NEC (i.e., on frame length). Both OP-OSEM and SP-OSEM showed less bias in activity concentration than did other iterative methods. However, they still showed differences in activity concentration distribution as a function of NEC, with a 75% change in the GM-to-WM contrast ratio between long- and short-duration frames (P < 0.001). This is similar for most iterative methods with changes of 92% (OSWLS, P < 0.001) up to 105% (ANW-OSEM, P < 0.001). In all cases 3D-FBP provided the most accurate results, independent of the underlying noise level (with a maximum GM-to-WM contrast ratio error of 9%, which was not statistically significant; P > 0.5, except 30 s [P > 0.05]). However, 3D-FBP showed a large SD of up to 41% in GM-to-WM contrast ratio at low NEC, whereas iterative reconstruction methods showed less SD at low NECs (<10%).

Hoffman Phantom: Decay Study

Supplemental Figure 2 shows results obtained from the ¹¹C-decay study. Most reconstruction methods had a constant GM-to-WM contrast ratio with decreasing NECs (between 8% and 10% maximum error in the GM-to-WM contrast ratio), except for ANW-OSEM, which showed a considerable change of 20% in the GM-to-WM contrast ratio. OSWLS (0%−4% difference), OP-OSEM (1%−6% difference), and SP-OSEM (3%−7% difference) showed good agreement with the first observed 3D-FBP GM-to-WM contrast ratio (2.34).

Clinical Example

For a single patient, time–activity curves reconstructed with various techniques are shown in Supplemental Figure 3. ANW-OSEM showed a bias of 10%−61% in GM, 24%−109% in WM, and 26%−100% in the pons for short-duration frames (15–60 s). Other iterative reconstruction techniques reduced bias in the GM, WM, and pons to within 3.9%, 15%, and 11%, respectively. OSWLS showed the best correspondence with the 3D-FBP pons time–activity curve (<8%). Axial parametric images are shown in Figure 5. Images of 3D-FBP are noisy, especially for K₁ parametric data (and for R₁ data; data not shown). Therefore, no good comparison could be made between 3D-FBP and 3D iterative reconstruction techniques for K₁ and R₁. Visually, ANW-OSEM, compared with the other reconstruction methods, showed a clear underestimation in BP_ND. OSWLS showed the fewest artifacts. Correlation between several parametric analyses are summarized in Tables 1 and 2. All results are compared with 3D-FBP parametric data. V_T (Table 1) showed a slight positive bias for ANW-OSEM with respect to 3D-FBP (slope with intercept fixed at origin, 1.03), whereas other iterative reconstruction methods showed a slight negative bias (slope with fixed intercept at origin, 0.92–0.93). Using more than the recommended number of iterations did not show significant improvement (P > 0.05), whereas a lower number of iterations showed poorer performance (P < 0.05). For BP_ND (Table 2), ANW-OSEM showed a considerable negative bias with respect to 3D-FBP (slope with intercept fixed at origin, 0.48) and low correlation (Pearson correlation coefficient [r] = 0.91). Both OP-OSEM and SP-OSEM showed less bias with respect to 3D-FBP (slope with fixed intercept, 1.12–1.14) and a higher correlation (r = 0.96). OSWLS showed the best correspondence with 3D-FBP (slope with intercept fixed at origin, 1.02) and highest correlation (r = 0.99). Using a higher-than-recommended number of iterations for OSWLS showed only a slight improvement relative to 3D-FBP (slope with fixed intercept, 1.00; r = 0.98, P < 0.001).

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 5. 

Parametric V_T, K₁, and BP_ND images for human ¹¹C-flumazenil study. Each column represents a different reconstruction method. BFM = basis function method; RPM = receptor parametric mapping.

Convergence

A GM-to-WM contrast ratio comparable to the one for 3D-FBP was found as an acceptable compromise between computation time, GM and WM convergence, resolution effects (due to partial-volume effects), and noise. A further increase in the GM-to-WM contrast ratio would imply an increase in bias in GM and WM, in number of iterations (computation time), and in noise. Results from the inserts phantom experiment indicated that convergence was reached at the same number of iterations (Figs. 1 and 2) as was observed using the Hoffman phantom data (Supplemental Fig. 1). Both ANW-OSEM GM and WM converged to within 1%, compared with 3D-FBP, using 10 iterations. The convergence rate of OP-OSEM was about 1.6 times slower than that of ANW-OSEM, which is slightly better than what has been reported in a previous study using the manufacturer's OP-OSEM (4), in which OP-OSEM was said to converge about 2 times more slowly than ANW-OSEM. Convergence was reached for OP-OSEM using 16 iterations. This convergence is 2 times slower than the version used by the manufacturer, which converges after 3–11 iterations (7), in which 8 iterations were found to be optimal (9) when using the same criteria as used in this study. SP-OSEM also requires 1.6–1.7 times more iterations than does ANW-OSEM, which is slightly faster than the number of iterations reported in previous studies (17). In these 3D studies performed on an ECAT EXACT HR+ scanner (CTI/Siemens), 15 iterations were used for SP-OSEM and 4 for ANW-OSEM. OSWLS converged to an acceptable GM and WM activity concentration after 7 iterations. Although the number of iterations required for sufficient convergence was similar for both the Hoffman and the inserts phantom studies, clinical data were analyzed for various numbers of iterations to further study and demonstrate convergence properties.

Phantom Studies

3D-FBP showed good correspondence to the true activity concentration in short-duration frames of the homogeneous phantom (<2%). Although 3D-FBP showed some bias in cold and hot inserts, observed 3D-FBP biases were less dependent on NEC than biases seen with iterative methods (Supplemental Figs. 1 and 2; Figs. 3 and 4). However, these biases disappeared using the same phantom with contrasts similar to the Hoffman phantom (1:2; data not shown). Moreover, 3D-FBP provided a GM-to-WM contrast independent of NEC (Fig. 4C; Supplemental Fig. 2) in the Hoffman phantom studies, which resemble clinically relevant activity concentration distributions. Deviations in the GM-to-WM contrast were not statistically significant (P > 0.5). 3D-FBP was, therefore, chosen as a reference reconstruction method for further evaluations of Hoffman phantom and clinical studies.

OP-OSEM showed the lowest positive bias (<8%), but COV (<5.1·10²%) was twice as high as that for OSWLS (<2.2·10²%). In addition, OSWLS showed almost linearly increasing COV at low-frame durations, whereas this variation was larger for all other methods. However, OSWLS showed a higher positive bias (<15%) than did OP-OSEM. The performance of SP-OSEM was similar to that of OP-OSEM, as can be seen from Figure 3. This result is consistent with data from Ahn and Fessler (18), in which SP-OSEM only slightly improved bias over OP-OSEM in 2D simulations. However, as also observed previously (19,32) for simulated 2D transmission and emission scans, SP-OSEM, compared with OP-OSEM, reduced SD up to 11%.

All methods showed similar performance for the Hoffman phantom, with various frame durations, although both OP-OSEM and SP-OSEM showed a lower bias (<75%, P < 0.001) in GM-to-WM contrast ratio between short- and long-duration frames. For the Hoffman phantom decay study, ANW-OSEM performed poorly, with a change in the GM-to-WM contrast ratio of 20%. In a previous study (9), we reported a high-contrast change in the GM-to-WM contrast ratio (54%) using the manufacturer's implementation of OP-OSEM. Using the in-house–implemented OP-OSEM with the same convergence criteria, the same ROI definitions, and the same scan of the Hoffman phantom, we obtained a good (and stable) GM-to-WM contrast ratio (10%), as we did with most other reconstruction methods (between 8% and 10%). SP-OSEM, OP-OSEM, and especially OSWLS showed good agreement with 3D-FBP GM-to-WM contrast ratios (<7% difference).

Human Brain Study

ANW-OSEM showed considerable bias (<109% in WM) for short-duration frames (15–60 s) or low NEC. Other iterative reconstruction techniques reduced bias (<15%), which is consistent with the phantom studies (Figs. 1–4⇑⇑⇑; Supplemental Fig. 1).

Parametric images, reconstructed with 3D-FBP, were noisy, consistent with the phantom data in which a high COV (<1.5·10³%) was observed in the short-duration frames of the homogeneous phantom. Therefore, 3D-FBP is not recommended routinely for quantitative brain studies and should be used only to validate iterative reconstruction methods.

Parametric data derived from ANW-OSEM showed that bias had a large effect on parametric results, especially for reference-tissue models in which large underestimations were observed. This is primarily due to bias in the pons time–activity curve (Supplemental Fig. 3C). Consequently, reducing bias, currently observed in iterative reconstruction techniques at low frame durations, is essential for quantitative brain studies.

Visually, OSWLS showed the fewest artifacts in the parametric images. In general, OSWLS showed better correspondence to parametric results obtained with 3D-FBP, especially for reference-tissue models (Table 2; slope with intercept fixed at origin, 1.02; r = 0.99). This result is also consistent with phantom data and the data of Boellaard et al. (11). On the basis of correspondence between GM-to-WM contrast ratios (within 4% when compared with 3D-FBP) over different NECs, low bias in GM regions (<11%, P < 0.001), linearly increasing COV, and good correspondence with the 3D-FBP parametric data (slope with intercept fixed at origin, 1.02 [reference tissue] and 0.93 [plasma input]; r = 0.97–0.99), OSWLS is recommended for 3D HRRT brain scans.

Additional Remarks

¹¹C-flumazenil has the advantage of high extraction and therefore good scan statistics. Use of other tracers with lower extraction, for example, ¹¹C-PK11195, most likely will result in higher bias due to poorer scan statistics. Some bias is still observed for iterative reconstruction methods in short-duration frames that could influence parametric data analysis and could be worse for tracers with low extraction, especially for reference-tissue models in which reference tissue has a low uptake (no specific binding). Also, bias related to contrast, as is still observed for all iterative reconstruction methods, could cause problems with image-derived input functions (33). Therefore, future research should be directed at removing systematic bias, for example, by using iterative reconstructions that allow negative image values, such as negative maximum likelihood (34).

The resolution of clinical scanners will be improved over the next few years. These scanners will therefore likely contain large and sparse sinograms as a result of using smaller crystals and, consequently, may suffer from bias when using iterative reconstructions with nonnegativity constraints. This work may, therefore, be important for future high-resolution PET scanners as well.