Abstract
Matching images of different modalities can be achieved by the maximization of suitable statistical similarity measures within a given class of geometric transformations. Handling complex, nonrigid deformations in this context turns out to be particularly difficult and has attracted much attention in the last few years. The thrust of this paper is that many of the existing methods for nonrigid monomodal registration that use simple criteria for comparing the intensities (e.g. SSD) can be extended to the multimodal case where more complex intensity similarity measures are necessary. To this end, we perform a formal computation of the variational gradient of a hierarchy of statistical similarity measures, and use the results to generalize a recently proposed and very effective optical flow algorithm (L. Alvarez, J. Weickert, and J. Sánchez, 2000, Technical Report, and IJCV 39(1):41–56) to the case of multimodal image registrationOur method readily extends to the case of locally computed similarity measures, thus providing the flexibility to cope with spatial non-stationarities in the way the intensities in the two images are related. The well posedness of the resulting equations is proved in a complementary work (O.D. Faugeras and G. Hermosillo, 2001, Technical Report 4235, INRIA) using well established techniques in functional analysis. We briefly describe our numerical implementation of these equations and show results on real and synthetic data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alvarez, L., Weickert, J., and Sánchez, J. 2000. Reliable estimation of dense optical flow fields with large displacements. Technical Report, Cuadernos del Instituto Universitario de Ciencias y Tecnologías Cibernéticas. A revised version has appeared at IJCV 39(1):41–56.
Bosq, D. 1998. Nonparametric Statistics for Stochastic Processes. 2nd edn. vol. 110 of Lecture Notes in Statistics. Springer–Verlag: Berlin.
Cachier, P. and Pennec, X. 2000. 3d non-rigid registration by gradient descent on a gaussian weighted similarity measure using convolutions. In Proceedings of MMBIA, pp. 182–189.
Chefd'hotel, C., Hermosillo, G., and Faugeras, O. 2001. A variational approach to multi-modal image matching. In IEEE Work-shop on Variational and Level Set Methods, University of BritishColumbia, Vancouver,Canada. IEEE Computer Society, pp. 21–28.
Christensen, G., Miller, M., and Vannier, M. 1994. A 3d deformable magnetic resonance textbook based on elasticity. In Proceedings of the American Association for Artificial Intelligence, Symposium: Applications of Computer Vision in Medical Image Processing.
Courant, R. 1946. Calculus of Variations. New York.
Deriche, R. 1990. Fast algorithms for low-level vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1(12):78–88.
Evans, L. 1998. Partial Differential Equations. In Proceedings of the American Mathematical Society. vol. 19 of Graduate Studies in Mathematics.
Faugeras, O. and Keriven, R. 1998. Variational principles, surface evolution, PDE's, level set methods and the stereo problem. IEEE Transactions on Image Processing, 7(3):336–344.
Faugeras, O.D. and Hermosillo, G. 2001. Well-posedness of eight problems of multi-modal statistical image-matching. Technical Report 4235, INRIA.
Gaens, T., Vandermeulen, F.M.D., and Suetens, P. 1998. Non-rigid multimodal image registration using mutual information. In First International Conference on Medical Image Computing and Computer-Assisted Intervention. G. Goos and J. Hartmanis, (Eds.), vol. 1496 of Lecture Notes in Computer Science. Springer: Berlin.
Hata, N., Dohi, T., Warfield, S.,, Kikinis, R., and Jolesz, F.A. 1998. Multi-modality deformable regristration of pre-and intra-operative images for mri-guided brain surgery. In First International Conference on Medical Image Computing and Computer-Assisted Intervention. G. Goos and J. Hartmanis (Eds.), vol. 1496 of Lecture Notes in Computer Science. Springer: Berlin.
Hermosillo, G. 2002. Variational Methods for Multimodal Image Matching. PhD thesis, INRIA, The document is accessible at ftp://ftp-sop.inria.fr/robotvis/html/Papers /hermosillo:02.ps.gz.
Koenderink, J.J. and van Doorn, A.J. 1999. Blur and disorder. In Scale-Space Theories in Computer Vision, Second International Conference, Scale-Space'99, M. Nielsen, P. Johansen, O.F. Olsen, and J. Weickert (Eds.), vol. 1682 of Lecture Note in Computer Science, Springer: Berlin, pp.1–9.
Leventon, M. and Grimson, W. 1998. Multi-modal volume registration using joint intensity distributions. In Medical Image Computing and Computer-Assisted Intervention-MICCAI'98, W. Wells, A. Colchester, and S. Delp (Eds.). Cambridge, MA, USA, vol. 1496 in Lecture Notes in Computer Science. Springer: Berlin.
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., and Suetens, P. 1997. Multimodality image registration by maximization of mutual information. IEEE Transactions on Medical Imaging, 16(2):187–198.
Maintz, J., Meijering, H., and Viergever, M. 1998. General multi-modal elastic registration based on mutual information. In Medical Imaging 1998—Image Processing, vol. 3338, SPIE, pp. 144–154.
Meyer, C., Boes, J., Kim, B., and Bland, P. 1998. Evaluation of control point selection in automatic, mutual information driven, 3d warping. In First International Conference on Medical Image Computing and Computer-Assisted Intervention, Proceedings,G. Goos and J. Hartmanis (Eds.), vol. 1496 of Lecture Notes in Computer Science.
Nagel, H. and Enkelmann, W. 1986. An investigation of smoothness constraint for the estimation of displacement vector fiels from images sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8:565–593.
Netsch, T., Rosch, P., van Muiswinkel, A., and Weese, J. 2001. Towards real-time multi-modality 3d medical image registration. In Proceedings of the 8th International Conference on Computer Vision, Vancouver, Canada. IEEE Computer Society, IEEE Computer Society Press.
Parzen, E. 1962. On the estimation of probability density function. Ann.Math.Statist., 33:1065–1076.
Roche, A., Guimond, A., Meunier, J., and Ayache, N. 2000a. Multi-modal elastic matching of brain images. In Proceedings of the 6th European Conference on Computer Vision, Dublin, Ireland.
Roche, A., Malandain, G., and Ayache, N. 2000b. Unifying maximum likelihood approaches in medical image registration. International Journal of Imaging Systems and Technology: Special Issue on 3D Imaging, 11(1):71–80.
Roche, A., Malandain, G., Pennec, X., and Ayache, N. 1998a. The correlation ratio as new similarity metric for multimodal image registration. In Medical Image Computing and Computer-Assisted Intervention-MICCAI'98, W. Wells, A. Colchester, and S. Delp (Eds.), Cambridge, MA, USA, vol. 1496 in Lecture Notes in Computer Science. Springer: Berlin, pp. 1115–1124.
Roche, A., Malandain, G., Pennec, X., and Ayache, N. 1998b. Multi-modal image registration by maximization of the correlation ratio. Technical Report 3378, INRIA.
Rückert, D., Hayes, C., Studholme, C., Summers, P., Leach, M., and Hawkes, D. 1998. Non-rigid registration of breast mr images using mutual information. In Medical Image Computing and Computer-Assisted Intervention-MICCAI'98, W. Wells, A. Colchester, and S. Delp (Eds.), Cambridge, MA, USA, vol. 1496 in Lecture Notes in Computer Science. Springer: Berlin.
Toga, A., (Ed.). 1998. Brain Warping. Academic Press.
Trouvé, A. 1998. Diffeomorphisms groups and pattern matching in image analysis. International Journal of Computer Vision, 28(3):213–21.
Viola, P. and Wells, III, W. M. 1997. Alignement by maximization of mutual information. The International Journal of Computer Vision, 24(2):137–154.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hermosillo, G., Chefd'Hotel, C. & Faugeras, O. Variational Methods for Multimodal Image Matching. International Journal of Computer Vision 50, 329–343 (2002). https://doi.org/10.1023/A:1020830525823
Issue Date:
DOI: https://doi.org/10.1023/A:1020830525823