CoRX-NET: a Novel Unsupervised Deep Image Registration Network with Feature Sharing for Accurate Voxel-based Dosimetry

Introduction: Generating 3D time-integrated activity (TIA) maps in the voxel-based dosimetry for radionuclide therapy requires non-linear registration of SPECT/CT images acquired at different time points. Most widely used approach for SPECT/CT registration and TIA map generation is to apply an affine transformation matrix and nonlinear deformation fields generated from a pair of CT images to the SPECT images. However, in this conventional approach, the accuracy of SPECT image registration and TIA map generation is not high enough for tumor regions with insufficient morphological information in CT images. Therefore, in this study, we propose CoRX-NET, a novel unsupervised deep-learning based approach that simultaneously uses SPECT and CT images for accurate image registration required for voxel-based dosimetry.

Methods: For network training, 22 SPECT/CT image sets from 7 patients treated with ¹⁷⁷Lu-DOTATATE at Seoul National University Hospital were used. Each image set consists of 3 or 4 SPECT/CT images acquired at different time points. 468 pairs sampled from the image sets were divided into 408, 48, and 12 pairs for network training, evaluation, and internal test, respectively. Each training pair consists of two SPECT/CT images (i.e. fixed and moving SPECT/CT images) acquired at different times. The ¹⁷⁷Lu-DOTATATE SPECT/CT images acquired from 14 patients treated at Malaysia Sunway Medical Centre were used as the external test sets and the number of image pairs was 28. Swin-Transformer (CVPR 2021) was used as a backbone network and cross-stitch layer (CVPR 2016) was applied to share features generated from SPECT and CT images. LNCC (local normalized cross-correlation) loss was used for the similarity measurement of CT and SPECT, respectively. In addition, a bending energy regularization loss term was added to avoid image folding artifacts. The network performance was compared to Elastix and TransMorph (MIA 2022using L1 loss and SSIM of CT, SSIM of normalized SPECT, and LNCC of SPECT as metrics. Furthermore, the voxel-wise RMSE of the TIA map fitted with mono-exponential function was calculated for the comparison of the goodness of fitting.

Results: The comparison of metrics included in an ablation study is summarized in Table 1. The ablation study has shown that attaching a cross-stitch layer to the network results in better registration with less CT L1 loss and SPECT LNCC and higher SSIM of SPECT and CT. CoRX-NET showed more accurate registration performance compared to the Elastix and TransMorph, except for CT registration of external test sets. Although Elastix showed better registration performance for CT than CoRX-NET on external test sets, the deformed CT is not used directly for TIA map generation. The mean RMSEs of TIA maps in tumor and kidney were 178.60 and 57.13 for Elastix and 100.32 and 43.54 for CoRX-NET. CoRX-Net takes 6 seconds for registration whereas Elastix takes 50 seconds.

Conclusions: The proposed network showed better performance in sequential SPECT/CT registration compared to the conventional iterative registration method (Elastix) and TransMorph, which has shown high performance in medical image registration. As a result, a more reliable TIA maps could be generated using CoRX-NET, mitigating errors in voxel-based dosimetry due to mis-registration of SPECT images. Practical voxel-based dosimetry will be also possible with reduced registration time and a convenient end-to-end registration process.

• Download figure
• Open in new tab
• Download powerpoint

• Download figure
• Open in new tab
• Download powerpoint