|
|
||||||||
Basic Science Investigations |
1 Diagnostic Radiology and Nuclear Medicine Department, University of Western Ontario, London, Ontario, Canada
2 Department of Imaging, Cedars-Sinai Medical Center, Los Angeles, California
3 Department of Nuclear Medicine, Center for PET, Zentralklinik Bad Berka, Bad Berka, Germany
| ABSTRACT |
|---|
|
|
|---|
Key Words: thoracic PET image registration multimodality image fusion nonlinear registration
| INTRODUCTION |
|---|
|
|
|---|
Image registration methods can use fiducial markers or exploit inherent image similarities (3) to achieve data alignment. Techniques based on fiducial markers are especially useful in brain image registration (4) because the rigid character of the skull allows relatively accurate positioning of the markers and, therefore, good reproducibility and accuracy. The use of fiducial markers, however, may significantly complicate image acquisition procedures, landmark-based registration cannot be performed retrospectively, and it may have limited accuracy (5). In the registration of the thoracic and abdominal area, the placement and accuracy of external fiducial markers would be problematic because of soft-tissue deformations of internal organs such as the lungs. The logistic difficulties in using fiducial landmarks for all patients prospectively limit the practicality of this approach.
Image-based algorithms for multimodality registration have been developed and applied primarily to brain images from various modalities (2,3). These algorithms can be divided into surface-matching methods and voxel similarity methods. Voxel similarity methods have been demonstrated to have superior performance in comparison with surface matching or the external fiducial landmarkbased methods in the registration of PET CT and MRI images of the brain (4,6). Some registration algorithms have been applied in multimodality fusion of images of the thorax and abdomen in preliminary studies. The multimodality registration of CT and PET or SPECT of the thorax has been approached as a rigid-body problem using a surface-based technique based on chamfer matching (7), lung segmentation (8,9), interactively defined homologous markers (10), and mutual information (MI) maximization (11).
Nonlinear approaches, which attempt to compensate for soft-tissue deformations, have been proposed for registration of the thoracic and abdominal regions by maximizing MI based on the 15-parameter thin-plate-spline (TPS) model (12) or piecewise linear approach to independently move portions of the volume (13). A nonlinear polynomial approach has been used to create 3-dimensional (3D) abdominal atlases of MRI (14) and for MRI/CT matching (15). Image warping is typically accomplished by energy-minimizing interpolation such as TPSs (16) or B-splines (17). The automation of nonlinear registration techniques is of great importance because the number of adjustable image parameters can be very large. The limitations of the surface-based nonlinear registration techniques are that they are sensitive to the initial errors in the image segmentation phase, whereas the computational cost of the more robust, volume-based iterative approach can be prohibitive.
In this study, we developed software algorithms for thoracic PET/CT image registration, which would allow robust, rapid, fully automated, and retrospective fusion of images obtained with stand-alone PET and CT machines. We designed and implemented such an algorithm for thoracic PET/CT registration, overcoming the difficulty of incomplete anatomy on PET emission data by using both emission and transmission datasets. Two complementary steps were investigated: (a) a linear 9-parameter registration, which alone may be suitable for image registration, if both scans have been acquired during tidal breathing; and (b) a nonlinear registration refinement, for the correction of the thoracic deformation during the breath-hold CT acquisition. We put particular emphasis on the performance of the software algorithm to achieve practical computational times for clinical deployment.
| MATERIALS AND METHODS |
|---|
|
|
|---|
PET was performed with a whole-body PET scanner (ECAT EXACT; Siemens CTI) with an axial field of view of 15.8 cm, resulting in 47 transverse slices with a slice thickness of 3.4 mm. 18F-FDG of high specific activity was produced in the radiopharmacy of the Bad Berka PET Center. All patients were asked to fast for 12 h (overnight) before the study to minimize glucose utilization by normal tissue and to ensure standardized glucose metabolism. To reduce the confounding effects of changing bladder activity on assessment of the pelvis, patients were encouraged to drink water (750 mL) before the study and were given 20 mg furosemide 30 min before imaging. The images were acquired approximately 60 min after the administration the intravenous administration of 6 MBq 18F-FDG per kilogram of body weight. All scans consisted of 46 bed positions. Patients were placed in the supine position with arms elevated. Transmission scans were obtained with 68Ge rod sources (4 min increasing up to 6 min, depending on the activity of the rod sources). Emission data (6- to 8-min acquisition times) corrected for random events, dead time, and attenuation were reconstructed by means of iterative processing of both emission and transmission data by the ordered-subsets expectation maximization algorithm. Transmission data was reconstructed with the filtered backprojection technique. The image pixel counts were calibrated to activity concentration (Bq/mL) and were decay corrected using the time of tracer injection as the reference. The resulting in-plane image resolution of the transaxial images was approximately 8-mm full width at half maximum (FWHM), with an axial resolution of approximately 5-mm FWHM. Pixel sizes of reconstructed images was 4.34 mm for emission data and 5.15 mm for transmission data (due to different reconstruction methods) but were equalized later by the registration software. Transmission and emission scans had the range of 85168 slices. Both transmission and emission matrix sizes were 128 x 128.
Initial Data Processing
After the reconstruction, images were transferred to the HERMES (Nuclear Diagnostics) image workstation. Before image registration, the CT data were resampled to the 256 x 256 matrix, and the volume extents and pixel sizes were adjusted automatically by our registration software to equalize PET and CT field-of-view extents. Transmission and attenuation-corrected emission PET pairs were considered for the image registration with CT datasets. The linear module of the presented registration algorithm has been implemented on the HERMES platform as a module in the Multimodality Fusion toolkit. No manual operation was performed during the data preprocessing steps.
Registration Algorithm
The algorithm consists of several steps described below and a general overview is presented in Figure 1.
|
Transformed PET data (both emission and transmission) and CT voxel intensities can be defined as random variables with marginal probability distributions pCT(c) and pPET(et) and a joint probability distribution pPETCT(et,c), where et are PET emission and transmission intensity values and c are the CT voxel intensity values; then the MI(PET,CT) between PET and CT, is defined as:
![]() | (Eq. 1) |
![]() | (Eq. 2) |
To improve the computational efficiency and reduce the effect of the random noise in the calculation of the cost function, we reduced the number of the intensity levels to 100 in each modality and, therefore, the number of all intensity pairs was limited to 10,000, similar to our previous work (22). This reduction in the number of intensity bins assumes that the intensity variations of <1% are discarded in the calculation of the MI. The use of a larger number of bins did not have any significant effect on the registration as assessed in our preliminary evaluations. The joint probability is subsequently calculated by dividing the number of such voxel pairs (#) by the number of all voxels in the volume:
![]() | (Eq. 3) |
The marginal probability distributions pPET(et) and pCT(c) are defined as the probabilities of encountering a particular intensity in each of the volumes (PET or CT), regardless of the corresponding voxel intensities in the other volume. They can be calculated as follows:
![]() | (Eq. 4) |
![]() | (Eq. 5) |
Transmission maps acquired during PET scans for the purposes of attenuation correction can be used for the image registration (23,24). To reduce the dependence on the highly variable PET image characteristics, we have defined the cost function as a function of both transmission and emission scans, assuming both have been acquired in the same orientation. We have evaluated the registration using emission data alone (EM), transmission data alone (TR), and also using a combination of both emission and transmission data (TR+EM). In this implementation, the combination was achieved by addition of 2 volumes after normalization with a particular weighting function. To determine a suitable weighting factor we have attempted to maximize the number of successful registrations after image misalignment testing 3 different factors. The maximum number of successful registrations (95/100) was obtained for the weighting factor of 0.8 for emission and 0.2 for the transmission scan. Therefore, this weighting value was used in all subsequent experiments.
Minimization Algorithm for Linear Registration.
The automatic linear registration used a simplex-based minimization algorithm (25) to maximize the MI measure as defined above. To be able to use the minimization algorithm for maximization of the MI, the cost function was defined to be -MI(PET,CT) or -NMI(PET,CT). In minimizing the cost function, the simplex algorithm adjusted translations along the 3 coordinate axis (X shift, Y shift, and Z shift) and rotations within the 3 orthogonal planes (XY rotation, XZ rotation, and YZ rotation) as well as X, Y, and Z scalings, similar to the approach used by Hill et al. for correction of pixel scaling errors (26). In each iteration, PET volumes were resampled using trilinear interpolation. The iteration process was terminated when the relative decrease in the cost function was below a predefined threshold (<0.1%).
To improve the performance and reduce the possibility of entrapment in local minima, we used a multiresolution approach. The minimization process was repeated for PET and CT volumes rebinned to several matrix sizes, starting from the 32 x 32 matrix size. The search range was then progressively reduced for each cycle at a given image resolution. We evaluated the PET/CT registration algorithm with a different number of multiresolution steps.
Nonlinear Algorithm.
After linear registration with anisotropic linear scaling, we performed an additional step to correct for any remaining nonlinear differences. Lung voxels were automatically identified on PET transmission and CT volumes after linear registration using an adaptive threshold region-growing technique. Emission PET data were not used in the warping process. Second, all voxels inside the patient body were classified as lung or soft-tissue voxels. Subsequently, corresponding points on PET and CT lung outlines and external body outlines were identified by automatic ray tracing from centroids of the lungs and outside the body (Fig. 2A). In addition, spine voxels were identified on CT scans and assumed to be in the same location on the transmission scans (where spine is typically not clearly identifiable). The corresponding point pairs (ray points intersecting the CT and PET lung and body surfaces) formed 3D displacement vectors, which were then used as an input to the image warping module (Fig. 2B). To reduce image artifacts due to the incorrectly and inconsistently defined point pairs, we apply median filtering of vectors based on local neighborhoods. We assume that for each motion vector there exists a spheric neighborhood of a given radius, in which all motion vectors must have similar directions. This step avoids image artifacts caused by dissimilar motion vectors positioned close to each other.
|
Evaluation of Algorithm
To test the reproducibility of various methods due to the entrapment in the local minima, we performed multiple registrations of the same data from different initial 3D orientations. The PET emission and transmission volume pairs were arbitrarily misaligned by 6 parameters: 3 angles varied from 0° to 10° in 2° steps, translations along the x- and y-axes varied in 5-pixel steps from 0 to 15 pixels (065 mm), and translation along the z-axis varied from 0 to 15 pixels in 5-pixel steps (075 mm). These misalignments were selected to represent a range that could be encountered in clinical practice. These tests were repeated for multiresolution and single-resolution techniques, for different similarity measures (MI and NMI), for the registration of emission and transmission pairs, and for the registration of emission or transmission data alone for 5 alphabetically selected datasets. Because the correct position was unknown in this experiment with clinical data, we defined the correct position to be the mean transformation for all registrations starting from different initial positions, which were labeled by the observer as "correctly aligned." The reproducibility error was subsequently defined as the mean of absolute differences from that correct position for each of the registration parameters. All registration results with various combinations of parameters (300 tests in total) were also visually assessed by 2 observers and failures were recorded. Visual success was defined if a 9-parameter transformation was close to the correct alignment, without considering the remaining nonlinear mismatch.
On the basis of the tests described in the previous section, the most optimal linear registration technique was chosen. Subsequently, the results of the linear and nonlinear registration for all original 18 datasets (15 breath-hold, 3 normal breathing) were evaluated by 2 observers. Linear registration was judged without considering the remaining nonlinear mismatches in the diaphragm or thoracic area. Warping results were assessed visually, separately by 2 observers, and also quantitatively by comparing the lung volumes determined from the inspiration CT scans and from the transmission PET scans before and after warping was applied. All computing times were recorded.
| RESULTS |
|---|
|
|
|---|
|
|
|
|
The linear registration of all original 18 datasets using MI, multiresolution, and EM+TR combination was considered to be aligned automatically as close as possible to the correct position by translation, rotation, and scalings, as judged by the 2 observers. In 8 cases (3 normal breathing, 5 breath-hold), there was no visible diaphragm misregistration after applying the 9-parameter transformation. In the remaining 10 cases, there were evident nonlinear misregistration errors in the diaphragm area that were further corrected by the nonlinear algorithm. After application of the nonlinear algorithm, all registrations were judged successful. The differences between the original unregistered and registered positions were up to 20 pixels. The biggest differences were in the Z direction. Scaling parameters were adjusted up to 115%, the largest amount also in the Z direction.
Figure 4A represents the results of the linear registration with the CT acquired during deep breath-hold. In this case, the misalignment in the diaphragm area is evident and 9-parameter registration could not correct for the diaphragm displacement. Note large differences in the axial field-of-view extent between CT and PET, which has not caused difficulties for the automated registration algorithm. Figure 4B shows an example of the linear registration in the case of both CT and PET acquired during normal breathing, where no remaining nonlinear differences are visible.
|
The median vector filtering technique eliminated the warping artifacts due to noncollinear warp vectors in close proximity to each other. Figure 5 illustrates the warping artifact before applying the median vector filtering (left) and the subsequent improvement of the alignment after the algorithm was applied (right). In addition to eliminating the artifacts, the filtering allowed reduction of the number of the point pairs, from the 270400 range before filtering to the 225365 range after filtering (on average, by 10%), which also resulted in the proportional decrease of the processing time. The nonlinear algorithm execution time is <30 s on a Pentium 2-GHz computer.
|
|
The results of the image registration before and after warping, for 2 typical cases, are shown in Figure 6 (CT/PET transmission overlay) and Figure 7 (CT/PET emission overlay). In these 2 cases, there is a large mismatch in the diaphragm area when only linear registration has been applied. The linear registration results in the PET emission liver activity being placed in the lung area after fusion (Fig. 7, left). The nonlinear correction corrects this effect, which is shown with the warped transmission map (Fig. 6B) and with warped emission data (Fig. 7, right).
|
|
| DISCUSSION |
|---|
|
|
|---|
An alternative to the nonlinear image registration approach is to perform multiple iterative warps by iteratively maximizing MI because our results indicate it increases after a correct warp. This approach, however, would require a massive amount of computing time because the number of warping parameters in our technique is very large (hundreds) in comparison with a 9-parameter search for the linear transformation. Others attempted to maximize measures of MI with a smaller amount of nonlinear parameters (12), but this number would not allow the smooth local deformations needed in this application. At the same time, we wanted to ensure that warping with such large number of parameters does not introduce spurious local deformations due to the noise in the data; therefore, we smoothed the warp vectors, using local median filtering.
The novel use of combined emission and transmission datasets allows a high degree of initial convergence and full automation of the algorithm. However, the final warping transformation is based on the transmission data alone, ensuring the independence of the functional uptake of the radiopharmaceutical. Including functional emission image in the warp determination could lead to errors, for example, if a part of a lesion visualized on a CT scan is necrotic and does not exactly match the CT anatomic volume.
This study has been applied to thoracic PET/CT scans, in which the full lung coverage was present on both transmission and CT scans; alternative strategies have to be designed for warping of abdominal or head and neck scans because the deformation may be more complex in these areas. In addition, the axial sampling of the CT in this study was 5 mm, which reduces the accuracy of the diaphragm position determination due to possible artifacts in that area. Furthermore, the ray-tracing algorithm to model the lung expansion is to some extent arbitrary and may not truly reflect the mechanical properties of the lungs. Further work in this area is required to establish the best deformation method of the lung that would closely follow the individual respiratory cycles. We also investigated only a subset of possible breathing patternsnamely, breath-hold and a limited number of normal breathing scans. Our approach would need to be validated in other situations, such as imaging during normal expiration or maximum expiration (29). The acquisition of the respiratory gated CT datasets may be required to validate the improvements in the software warping methods.
The principal difficulty in validating the proposed approaches is the lack of a gold standard to determine the algorithm success. Only indirect, " proxy" measures of the registration can be obtainedfor example, the algorithm convergence, reproducibility, and visual assessment. Typically, image registration algorithms have been validated with external fiducial markers in the brain but, in the case of soft tissue and internal organs, this approach is not feasible. We are currently investigating a validation approach that would use the data from the hybrid PET/CT scanner as the gold standard, with PET volumes arbitrarily misaligned using computer software before the re-registration. The validation could then assume the original PET/CT position to be the gold standard. The errors in the intrinsically fused hybrid scanner PET and CT data alignment, however, have been reported to be up to 11 mm in the diaphragm area (30), despite being collected during a single imaging session without apparent patient motion. Therefore, the use of such imperfectly aligned PET/CT data as a reference for the registration algorithm may result in incorrect estimation of the registration errors.
The evaluation of the linear algorithm suggests that the combination of the emission and transmission data, coupled with the multiresolution strategy and MI optimization, is the most effective in this application. Although normalized MI has been suggested as the more optimal technique in brain registration applications (21), we have found that this criterion resulted in less exhaustive search and more failures, when coupled with the simplex minimization algorithm, at least for the type of data tested. The multiresolution strategy allows better convergence and allowed a 95% success rate of the linear component despite of large misalignments (up to 15 pixels); this is due to the fact that initially a more rapid search is performed, allowing more extensive search of the alignment parameters. Use of the multiresolution step reduces the effect of the local-minima entrapment and increases the overall speed of the algorithm and, therefore, it should be always used.
We found that combination of the emission and transmission data results in the best convergence. Most previous approaches have been using exclusively one of the scans (emission or transmission) for the registration. The improved convergence can be most likely attributed to additional information present in both images. For example, spine and kidney structures are well visualized on the emission scans but are not visible on the transmission scans. On the other hand, the external body outlines and lungs are well visualized on the transmission scans. Further improvement of this technique may involve use of the multivariate image registration approach (31) to separate the emission and transmission volumes during the evaluation of the cost function.
The nonlinear image registration is difficult to validate quantitatively without a gold standard. Therefore, we relied primarily on visual verification of the results. One objective measure of the nonlinear alignment is the value of MI after the warp because this measure was not directly used during the warping process. The MI value increased in all cases, indicating that the warping technique applied after linear registration, although based on lung and body surfaces, further improves the statistical dependence of the coregistered volumes. Another quantitative measurement of the warp performance is the comparison of the lung volumes before and after the warp, as performed in this study (Table 4), indicating that lung volumes are similar after warp.
In the light of the recent introduction of PET/CT hybrid scanners (1), the software approach can be viewed from 2 perspectives. On the one hand, it allows automated fusion of PET and CT when only stand-alone PET and CT scanners are available, which can be the case in many departmental configurations. On the other hand, it allows merging of the data irrespective of the time of the acquisition, which is potentially a great benefit for the PET or PET/CT systems for follow-up data comparisons and for the registration with other modalities and additional CT scans. Although the presented method was applied to register and warp transmission data to the CT data, conceivably we could apply a similar technique to the registration of serial CT scans (because the ray tracing was performed on the segmented CT data). In addition, there may be situations when breath-hold CT in the inspiration phase is required and then PET/CT data may be acquired with a second low-dose CT, which would then need to be coregistered to the high-quality inspiration-phase CT using our approach. A future improvement of the registration could be accomplished by the respiratory gated PET coregistered to the inspiration CT (32). Similar registration techniques have been suggested for CT/MRI registration (33) and, therefore, it is likely that such an approach could be used for the registration of PET images to MRI and other modalities.
The automatic PET/CT image registration has several clinical applications. As reported in PET/CT studies, the use of image fusion changed the disease management in 20%30% of cancer patients (34,35). Coregistered data can also be used in radiation treatment planning applications. Recent reports suggest that planning of radiation therapy is significantly modified when coregistered data are used (36). Further applications of coregistered PET/CT imaging can be explored, such as CT/PET-guided biopsy or image-guided surgery.
| CONCLUSION |
|---|
|
|
|---|
| ACKNOWLEDGMENTS |
|---|
| FOOTNOTES |
|---|
For correspondence or reprints contact: Piotr J. Slomka, PhD, Department of Imaging, Cedars-Sinai Medical Center, #A047 8700 Beverly Blvd., Los Angeles, CA 90048.
E-mail: Piotr.Slomka{at}cshs.org
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
Y. Suzuki, P. J. Slomka, A. Wolak, M. Ohba, S. Suzuki, L. De Yang, G. Germano, and D. S. Berman Motion-Frozen Myocardial Perfusion SPECT Improves Detection of Coronary Artery Disease in Obese Patients J. Nucl. Med., July 1, 2008; 49(7): 1075 - 1079. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. W. Townsend Dual-Modality Imaging: Combining Anatomy and Function J. Nucl. Med., June 1, 2008; 49(6): 938 - 955. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Shiraishi, S. Tomiguchi, D. Utsunomiya, K. Kawanaka, K. Awai, S. Morishita, T. Okuda, K. Yokotsuka, and Y. Yamashita Quantitative Analysis and Effect of Attenuation Correction on Lymph Node Staging of Non-Small Cell Lung Cancer on SPECT and CT. Am. J. Roentgenol., May 1, 2006; 186(5): 1450 - 1457. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. Brechtel, M. Klein, M. Vogel, M. Mueller, P. Aschoff, T. Beyer, S. M. Eschmann, R. Bares, C. D. Claussen, and A. C. Pfannenberg Optimized Contrast-Enhanced CT Protocols for Diagnostic Whole-Body 18F-FDG PET/CT: Technical Aspects of Single-Phase Versus Multiphase CT Imaging J. Nucl. Med., March 1, 2006; 47(3): 470 - 476. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. S. Halpern, C. Schiepers, W. A. Weber, T. L. Crawford, B. J. Fueger, M. E. Phelps, and J. Czernin Presurgical Staging of Non-small Cell Lung Cancer: Positron Emission Tomography, Integrated Positron Emission Tomography/CT, and Software Image Fusion Chest, October 1, 2005; 128(4): 2289 - 2297. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Shekhar, V. Walimbe, S. Raja, V. Zagrodsky, M. Kanvinde, G. Wu, and B. Bybel Automated 3-Dimensional Elastic Registration of Whole-Body PET and CT from Separate or Combined Scanners J. Nucl. Med., September 1, 2005; 46(9): 1488 - 1496. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y. Nakamoto, S. Sakamoto, T. Okada, K. Matsumoto, E. Minota, H. Kawashima, and M. Senda Accuracy of Image Fusion Using a Fixation Device for Whole-Body Cancer Imaging Am. J. Roentgenol., June 1, 2005; 184(6): 1960 - 1966. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. M. Blodgett, B. Casagranda, D. W. Townsend, and C. C. Meltzer Issues, Controversies, and Clinical Utility of Combined PET/CT Imaging: What Is the Interpreting Physician Facing? Am. J. Roentgenol., May 1, 2005; 184(5_supp): S138 - S145. [Abstract] [Full Text] [PDF] |
||||
![]() |
J.-H. Kim, J. Czernin, M. S. Allen-Auerbach, B. S. Halpern, B. J. Fueger, J. R. Hecht, O. Ratib, M. E. Phelps, and W. A. Weber Comparison Between 18F-FDG PET, In-Line PET/CT, and Software Fusion for Restaging of Recurrent Colorectal Cancer J. Nucl. Med., April 1, 2005; 46(4): 587 - 595. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. J. Slomka, H. Nishina, D. S. Berman, X. Kang, C. Akincioglu, J. D. Friedman, S. W. Hayes, U. E. Aladl, and G. Germano "Motion-Frozen" Display and Quantification of Myocardial Perfusion J. Nucl. Med., July 1, 2004; 45(7): 1128 - 1134. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. V. Vogel, W. J.G. Oyen, J. O. Barentsz, J. H.A.M. Kaanders, and F. H.M. Corstens PET/CT: Panacea, Redundancy, or Something in Between? J. Nucl. Med., January 1, 2004; 45(90010): 15S - 24. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. J. Slomka Software Approach to Merging Molecular with Anatomic Information J. Nucl. Med., January 1, 2004; 45(90010): 36S - 45. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||