A large number of clinically important and medically difficult decisions in diagnostic radiology involve interpreting the information derived from multiple imaging modalities. This is especially true in the assessment of heart disease, wherein at least two types of image information are generally required prior to deciding on the course of action: structural information describing coronary vessel anatomy and functional information related to heart muscle physiology. This paper will present and discuss the methods and results associated with a research program aimed at quantifying and visualizing the unified anatomic and physiologic information obtained from these complementary imaging modalities. The discussions will emphasize the reconstruction, processing, and visualization of three-dimensional cardiovascular structure, including the procedures and results obtained from phantom and patient studies.