Joint Segmentation of Multi-Channel Images Using Multigraph Cuts

Objectives: Diagnostic and therapeutic tasks often rely on multiple images, such as those derived from multimodal, multi-time-frame (e.g. dynamic scanning), or multi-parametric (e.g. different MR pulse sequence based) acquisitions. Quantitative interpretation and analysis of such image groups or “channels” leads to the need for tools to jointly segment such multi-channel images. Our objective is to develop a joint segmentation approach based on multigraph cuts.

Methods: Our joint segmentation approach is based on unified cuts across a multigraph, defined as a composite of graphs generated from different image channels, thereby sharing the same node set but exhibiting different topologies. A composite normalized Laplacian for the multigraph is computed from the normalized Laplacians for each of the individual graphs. The unified cut is obtained by solving an optimization problem that seeks to ensure that the composite cut cost (based on the multigraph Laplacian) is minimized while ensuring similarity with each individual graph’s minimum cut. Minimum cuts for the individual graphs are first computed using a spectral clustering step. The segmentation problem is then posed as an eigenvalue decomposition problem based on the multigraph Laplacian. We solve the eigenvalue decomposition problem for the composite Laplacian, creating an effective cut that is constrained to be similar to each individual cut thus ensuring qualitative similarity between the unified cut and the individual cuts.

Results: The multigraph segmentation technique was first tested in digital phantom studies and then applied to the following clinical settings: 1) Dynamic imaging: Segmentation of liver lesions in hepatocellar carcinoma patients using dynamic [18F]FDG-PET. 2) Multi-parametric Imaging: Segmentation of glioblastoma multiforme (GBM) using MR images (FLAIR, FSE, T1 weighting, T1 weighting with Gd contrast). The joint segmentation was compared with single-channel segmentation results based on each individual graph’s minimum cut.

Conclusion: Multimodal, multi-time-frame, and multi-parametric imaging accumulates information from multiple structural and/or functional sources thereby facilitating diagnostic and therapeutic decisions. Joint segmentation based on unified multigraph cuts enables us to analyze data by combining disparate information sources. In comparison, single-channel segmentation approaches fail to capture the full-spectrum of anatomical/physiological information available. We have successfully applied the method to segment cancerous lesions in different clinical cases and contrasted the results with single-channel segmentation. Research Support: This work has been supported by NIH grants K01AG050711 and R01EB013293.

• Download figure
• Open in new tab
• Download powerpoint