Contrast enhancement in emission tomography by way of synergistic PET/CT image combination

doi:10.1016/j.cmpb.2007.12.009

Computer Methods and Programs in Biomedicine

Volume 90, Issue 3, June 2008, Pages 191-201

https://doi.org/10.1016/j.cmpb.2007.12.009 Get rights and content

Abstract

The display of image fusion is well accepted as a powerful tool in visual image analysis and comparison. In clinical practice, this is a mandatory step when studying images from a dual PET/CT scanner. However, the display methods that are implemented on most workstations simply show both images side by side, in separate and synchronized windows. Sometimes images are presented superimposed in a single window, preventing the user from doing quantitative analysis. In this article a new image fusion scheme is presented, allowing performing quantitative analysis directly on the fused images.

Methods

The objective is to preserve the functional information provided by PET while incorporating details of higher resolution from the CT image. The process relies on a discrete wavelet-based image merging: both images are decomposed into successive details layers by using the “à trous” transform. This algorithm performs wavelet decomposition of images and provides coarser and coarser spatial resolution versions of them. The high-spatial frequencies of the CT, or details, can be easily obtained at any level of resolution. A simple model is then inferred to compute the lacking details of the PET scan from the high frequency detail layers of the CT. These details are then incorporated in the PET image on a voxel-to-voxel basis, giving the fused PET/CT image.

Results

Aside from the expected visual enhancement, quantitative comparison of initial PET and CT images with fused images was performed in 12 patients. The obtained results were in accordance with the objectives of the study, in the sense that the organs’ mean intensity in PET was preserved in the fused image.

Conclusion

This alternative approach to PET/CT fusion display should be of interest for people interested in a more quantitative aspect of image fusion. The proposed method is actually complementary to more classical visualization tools.

Introduction

Multimodality imaging has become a mandatory exploration in many clinical applications. PET/CT hybrid scanners constitute today a necessary tool in diagnosis, treatment and staging of cancer [1], [2], [3]. The complementary information provided by this kind of dual imaging device allows revealing the physiological state of malignant tumours by PET while in the same time, the CT image offers anatomical accuracy through high-spatial resolution. One of the specific uses of PET/CT that currently encounters increasing interest is intensity-modulated radiotherapy (IMRT). Recent works tend to prove indeed that PET/CT-guided IMRT improves treatment planning while reducing tissue doses, for example in head and neck cancer [4].

Aside from the problem of spatial co-registration of PET and CT images, which permits their superposition on a voxel-to-voxel basis, the effective management of images in a day-to-day clinical use consists of their visualization. This step is of great importance since it allows comparing the images and making an accurate judgment according to functional and morphological complementarity. Two main types of visualization techniques exist [5]. In the first one, the two images are displayed side by side in two separate windows, with synchronized commands and cursors. This method of display presents the advantage of preserving information but the efficient visual comparison of structures remains difficult. The second visualization approach is the overlay of both images in a single window. Several approaches can be chosen but most of them require two look-up-tables, generally a grey level one for the CT image and a colour one for the PET image. An easy way to proceed is to display a voxel of each image alternately, like a mosaic. Another approach is to blend the images using a single look-up-table but then the intensity in a voxel is a weighted sum of PET and CT intensities in the given voxel. Recent works in this domain include multi-image voxel composting [6] in which the CT image is decomposed into several layers with different ranges of contrast adjustments, each one corresponding to a particular tissue (bone, lungs, soft tissues, etc.). These layers are then weighted and mixed together, finally been blended with the PET image.

In oncology staging or treatment planning it is of decisive importance to follow the evolution of both tumour activity and size. However, such quantitative measurements cannot be directly derived from the fused images since the intensity in a voxel is a mixture of corresponding PET and CT intensities. Even if the anatomical information is preserved to a certain extent, it is impossible to measure the PET intensity in a given region of interest. Moreover, the complementarity aspect of fusion display loses its interest in the sense that it is limited to visual inspection.

By enlarging the field of investigation, one may notice that image fusion concerns many fields, like geosciences [7], food safety [8], fingerprints analysis [9], biometric imaging [10] or forensic investigations [11]. Nevertheless, the objective of all these studies remains largely in the scope of visual enhancement of images without considering the quantitative aspect which is of outmost importance in medical imaging. Some general surveys have been also performed but mostly tackling without specific attention the medical aspect of image fusion algorithms [12].

In this article, we introduce a new fusion display scheme able to preserve the quantitative functional information provided by PET, and in the same time, able to maintain morphological details of the CT. The algorithm is based on multi-resolution analysis of the PET and CT images using wavelets. After presenting the theory and implementation of this method, we apply it on a number of clinical whole-body example image datasets and perform quantitative analysis to demonstrate its potential for preserving relevant information.

Section snippets

Basic theory on continuous wavelet transform (CWT)

For the sake of clarity, definitions are given for an 1D function f, but a more general theory can be found in [14]. The wavelet transform W of an 1D, real, square-integrable function f is defined by $W (a, b) = \int_{- \infty}^{+ \infty} f (x) ψ^{*} (\frac{x - b}{a}) d x,$ where a is the scale of the analysis and b is the parameter of translation corresponding to the position of the wavelet Ψ (Ψ^* stands for the complex conjugate of Ψ). W(a, b) is the inner product of f with the scaled and translated versions of Ψ: $W (a, b) = \int_{- \infty}^{+ \infty} f (x) ψ_{a, b}^{*} (x) d x = < f$

Design considerations

In this section we present the iterative “à trous” algorithm which can be easily implemented on a given image I. This discrete wavelet transform algorithm was introduced by Dutilleux [16], developed by Holdschneider [17] and detailed by Starck et al. [18]. The process gives an image sequence of coarser and coarser spatial resolution by performing successive convolutions with a low-pass filter h. At each iteration j, the spatial resolution of the approximation image app_j−1 is degraded to give

Alternative “à trous” implementation and combination process

The method relies on the fact that PET and CT images are spatially co-registered, i.e. both images can be superimposed and are reconstructed with the same voxel size. The fusion display approach presented here aims at preserving relevant information provided by each modality: anatomical details and high-spatial resolution from CT on the one hand, and functional data from the PET on the other hand. For this purpose the anatomical details provided by the CT and corresponding to resolution levels

Status report

An example of CT image decomposition using the “à trous” algorithm (version using dyadic then linear transformation) is shown in Fig. 2. The reconstructed CT image (Fig. 2g) is very similar to the original one (Fig. 2a) since their voxel-to-voxel difference (Fig. 2h) has only zero values apart from a limited number of voxels. The quantitative measurements on Fig. 2h give mean value 1.9 × 10⁻¹⁰ ± 7.4 × 10⁻⁷, min value −1.5 × 10⁻⁵ and max value 1.5 × 10⁻⁵ (mean of the absolute values 5.4 × 10⁻⁶ ± 5.7 × 10⁻⁶)

Lessons learned

In this article a new approach to PET/CT image fusion has been proposed for whole-body imaging. Contrary to the great majority of existing methods, the aim of the presented work was to provide the user with a fused image preserving both anatomical and functional data. The objective is therefore different from simply presenting two images in a visually convenient fusion display in the sense that quantitative analysis is also here considered as a possible step. In the proposed methodology the

Future plans

Concerning the algorithm, it would be of great interest to define a local model instead of a global one in order to modify the detail layers of the CT. This improvement could lead to the elimination of artifacts corresponding to structures present in the CT but not in the PET. The results presented in this study were obtained using 2D calculation only. Indeed, most discrete wavelet transforms still perform in 2D and 3D implementations do not exist or are not well validated yet. However, a

Conflict of interest statement

None declared.

References (19)

S. Senan et al.
Critical review of PET-CT for radiotherapy planning in lung cancer
Crit. Rev. Oncol. Hematol.
(2005)
N.D.S. Gani et al.
Remote sensing analysis of the Gorge of the Nile, Ethiopia with emphasis on Dejen-Gohatsion region
J. African Earth Sci.
(2006)
M.S. Kim et al.
Automated detection of fecal contamination of apples based on multispectral fluorescence images fusion
J. Food Eng.
(2005)
G.L. Marcialis et al.
Fingerprints verification by fusion of optical and capacitive sensors
Pattern Recogn. Lett.
(2004)
A. Noore et al.
Robust memory-efficient data level information fusion of multi-modal biometric images
Inform. Fusion
(2007)
F. Puente Leon
Automated comparison of firearm bullets
Forensic Sci. Int.
(2006)
H. Schöder et al.
PET/CT in oncology: integration into clinical management of lymphoma, melanoma and gastrointestinal malignancies
J. Nuclear Med.
(2004)
H.N. Wagner
Horizons near and far: molecular focus, global involvement
J. Nuclear Med.
(2005)
D.L. Schwartz et al.
FDG-PET/CT-guided intensity modulated head and neck radiotherapy: a pilot investigation
Head Neck
(2005)

There are more references available in the full text version of this article.

Cited by (25)

Color-appearance-model based fusion of gray and pseudo-color images for medical applications
2014, Information Fusion
Citation Excerpt :
Frequency encoding methods are developed to facilitate the interpretation of one modality by merging with details from the other [19]. Much effort has been made to retain the original information as much as possible, assorting to the multi-resolution analysis [20–22]. In our earlier research, we compared the performance of the multi-resolution analysis and proposed fused rules for biomedical applications [23,24].
Fusion of gray and pseudo-color images presents more information of biological tissues in a single image and facilitates the interpretation of multimodalities in medical practice. However, fused results are hampered by the problems of blurred details, faded color and artifact contours. This paper reports a method to solve the problems by precisely predicting the attributes of color perception using the color appearance model of International Commission on Illumination published in 2002 (CIECAM02). First, a rainbow palette is generated from the color attributes. It is uniform in lightness, and thus the valuable information of pseudo-color image can be totally sealed in its chromatic properties. Then the fusion process is carried out with the adjustment of gray image in lightness. Here, the predicted hue and saturation of pseudo-color image is merged with the predicted lightness of gray one. Therefore, information of two original images exists separately in achromatic and chromatic properties of the resulting image. Based on different color spaces (CSs) and color appearance models (CAMs), the color aggregations available for displaying fused images are presented and compared. The aggregation based on the CIECAM02 exhibited more uniform variation in lightness and hue. Fused results of simulated lesion and breast phantom manifested the compromise between the scope of gray and the perception of color. Furthermore, in the quantitative experiment on 49 sets of simulated ultrasound and strain images, the visual information fidelity (VIF) was applied to assess the similarity between the result and its sources. It revealed the superiority of the proposed method over the traditional ones including CSs-based methods, transparency technique, alternating display technique, frequency encoding methods, and maximum-selection-rule based rules. The results of two clinical cases demonstrated its practicality in medical applications. Besides, its feasibility in fusing two high-resolution structural images was preliminarily approved based on the simulated MRI data.
Human visual system inspired multi-modal medical image fusion framework
2013, Expert Systems with Applications
Multi-modal medical image fusion, as a powerful tool for the clinical applications, has developed with the advent of various imaging modalities in medical imaging. The main motivation is to capture most relevant information from sources into a single output, which plays an important role in medical diagnosis. In this paper, a novel framework for medical image fusion based on framelet transform is proposed considering the characteristics of human visual system (HVS). The core idea behind the proposed framework is to decompose all source images by the framelet transform. Two different HVS inspired fusion rules are proposed for combining the low- and high-frequency coefficients respectively. The former is based on the visibility measure, and the latter is based on the texture information. Finally, the fused image is constructed by the inverse framelet transform with all composite coefficients. Experimental results highlight the expediency and suitability of the proposed framework. The efficiency of the proposed method is demonstrated by the different experiments on different multi-modal medical images. Further, the enhanced performance of the proposed framework is understood from the comparison with existing algorithms.
Fourier-wavelet restoration in PET/CT brain studies
2012, Nuclear Instruments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment
Citation Excerpt :
The recent availability of combined PET–CT and PET–MRI scanning technology has increased the interest in such approaches, which use synergistically multimodal information to correct for PVE. Usually, these PVC methods rely on the availability of a segmented MRI image of the subject that is co-registered with the PET image [8,21–24] and they are prone to registration and segmentation errors. Such errors affect the precision of the recovery of the true radioactivity levels [4].
Our goal is to improve brain PET imaging through the application of a novel, hybrid Fourier-wavelet (WFT) restoration technique. The major limitation of PET studies is a relatively poor resolution in comparison with MRI and CT imaging and there is a need for improved PET imaging. A GE DLS PET/CT 16 slice system was used to acquire the studies. In order to create restoration filters the point source study was performed. The 6-fillable spheres and 3D Hoffman brain phantom studies were acquired and used to test and optimize the restoration approach. The patient data used in the study were acquired in a 3D PET mode, using the standard clinical protocol. Here, we have implemented Fourier-wavelet regularized restoration. In the Fourier domain, the inverse of modulation transfer function was multiplied by a Butterworth low-pass filter, order n=6 and cut-off frequency f=0.35 cycles/pixel. In addition, wavelet (Daubechies, order 2) noise suppression was applied by “hard threshold”. Hot spheres and 3D Hoffman brain studies showed that the restoration process not only improves resolution and contrast but also improves quantification in 3D PET/CT imaging. The average contrast increase was 19% and the quantification improved in the range 8−20% depending on sphere size. In the restored images, there was no significant increase in noise when compared with the original images. The clinical studies followed brain phantom findings, i.e., the restored images had better contrast and resolution properties, when compared with the original images. The results of the study demonstrate that the quality and quantification of 3D brain ¹⁸F FDG PET images can be significantly improved by Fourier-wavelet (WFT) restoration filtering.
Multiscaled combination of MR and SPECT images in neuroimaging: A simplex method based variable-weight fusion
2012, Computer Methods and Programs in Biomedicine
Citation Excerpt :
The transparency technique leads to darkened results due to the black background of functional images; the spatially interlacing loses original details aside from a low-contrast of the fused image; the temporally interlacing is a hardware-based dynamic method whose result is difficult to save for subsequent processing; the color space transform based methods have several problems, including the change of the color system, awful compromise between original images and the lack of interactive considerations; the segmentation based method faces the problems of threshold selection and sharp artificial edges in some applications. Except for the fusion for localization, some attempt to improve the definition of functional images [17–19] and others combine two structural images to obtain more comprehensive structural information [20,21]. Image fusion has also intrigued the research interest in remote sensing [22–27].
Single-photon emission computed tomography (SPECT) images alone are difficult to understand in diagnosis, since anatomical structures are absent from the data. Studies on combination attempt to locate functional changes of the SPECT image by the magnetic resonance (MR) image. Due to the low similarity between original images, fused results are always darkened, obscured or loss some crucial anatomical structures. This paper has solved these problems by the variable-weight matrix which is estimated by minimizing the cost function using the simplex method. Under the generalized intensity-hue-saturation (GIHS) framework, the multiscaled analysis is presented for a better detail preservation. Besides, interactive approaches are discussed for the gradual variation between original images and the control of detail performance. The similarity assessment evaluates several different methods on a normal brain atlas. Two clips show the interactive property of the proposed method, while two medical cases demonstrate its clinical values. We conclude that the proposed method is superior to traditional methods, when considering the definition and the information capacity of fused results.
Biological image fusion using a NSCT based variable-weight method
2011, Information Fusion
Citation Excerpt :
However, the difficulty exists in selecting the proper criteria consistent with the subjective assessment of the image quality. The root mean square error (RMSE), correlation coefficients (CC) and mutual information (MI) are traditionally used [4,5,13], but they have little resemblance to the visual effect of the image similarity. In the recent study [26], large-scale subjective experiments assess the visual information fidelity (VIF), a novel image similarity criterion, and proves it to be a good substitution for the subjective assessment.
Biological image fusion, as a powerful tool for the protein study, has developed with the advent of various imaging modalities in molecular biology. Combining the fluorescent image and its corresponding phase contrast image will benefit the localization of the protein. However, resulting images of traditional methods are always difficult to compromise between multimodalities. This paper has solved this problem by a variable-weight fusion rule based on the nonsubsampled contourlet transform (NSCT). The intensity components of original images are combined in the multiscaled space and the fused image is obtained in the generalized intensity-hue-saturation (GIHS) frame. Validation experiments on 117 sets of Arabidopsis images are for two purposes: the comparison among different fusion rules and the impact of the multiscaled analysis in biological image fusion. Region-based quantified indexes reveal the similarity between fused images and original ones, and therefore demonstrate the superiority of the proposed method over traditional methods.
Functional and structural synergy for resolution recovery and partial volume correction in brain PET
2009, NeuroImage
Purpose: Positron Emission Tomography (PET) has the unique capability of measuring brain function but its clinical potential is affected by low resolution and lack of morphological detail. Here we propose and evaluate a wavelet synergistic approach that combines functional and structural information from a number of sources (CT, MRI and anatomical probabilistic atlases) for the accurate quantitative recovery of radioactivity concentration in PET images. When the method is combined with anatomical probabilistic atlases, the outcome is a functional volume corrected for partial volume effects.
Methods: The proposed method is based on the multiresolution property of the wavelet transform. First, the target PET image and the corresponding anatomical image (CT/MRI/atlas-based segmented MRI) are decomposed into several resolution elements. Secondly, high-resolution components of the PET image are replaced, in part, with those of the anatomical image after appropriate scaling. The amount of structural input is weighted by the relative high frequency signal content of the two modalities. The method was validated on a digital Zubal phantom and clinical data to evaluate its quantitative potential.
Results: Simulation studies showed the expected relationship between functional recovery and the amount of correct structural detail provided, with perfect recovery achieved when true images were used as anatomical reference. The use of T1-MRI images brought significant improvements in PET image resolution. However improvements were maximized when atlas-based segmented images as anatomical references were used; these results were replicated in clinical data sets.
Conclusion: The synergistic use of functional and structural data, and the incorporation of anatomical probabilistic information in particular, generates morphologically corrected PET images of exquisite quality.

View all citing articles on Scopus

View full text

Contrast enhancement in emission tomography by way of synergistic PET/CT image combination

Abstract

Methods

Results

Conclusion

Introduction

Section snippets

Basic theory on continuous wavelet transform (CWT)

Design considerations

Alternative “à trous” implementation and combination process

Status report

Lessons learned

Future plans

Conflict of interest statement

Crit. Rev. Oncol. Hematol.

J. African Earth Sci.

J. Food Eng.

Pattern Recogn. Lett.

Inform. Fusion

Forensic Sci. Int.

PET/CT in oncology: integration into clinical management of lymphoma, melanoma and gastrointestinal malignancies

J. Nuclear Med.

Horizons near and far: molecular focus, global involvement

J. Nuclear Med.

FDG-PET/CT-guided intensity modulated head and neck radiotherapy: a pilot investigation

Head Neck