Research ArticleCardiology

Deep Learning Analysis of Upright-Supine High-Efficiency SPECT Myocardial Perfusion Imaging for Prediction of Obstructive Coronary Artery Disease: A Multicenter Study

Julian Betancur, Lien-Hsin Hu, Frederic Commandeur, Tali Sharir, Andrew J. Einstein, Mathews B. Fish, Terrence D. Ruddy, Philipp A. Kaufmann, Albert J. Sinusas, Edward J. Miller, Timothy M. Bateman, Sharmila Dorbala, Marcelo Di Carli, Guido Germano, Yuka Otaki, Joanna X. Liang, Balaji K. Tamarappoo, Damini Dey, Daniel S. Berman and Piotr J. Slomka
Journal of Nuclear Medicine May 2019, 60 (5) 664-670; DOI: https://doi.org/10.2967/jnumed.118.213538

Abstract

Combined analysis of SPECT myocardial perfusion imaging (MPI) performed with a solid-state camera on patients in 2 positions (semiupright, supine) is routinely used to mitigate attenuation artifacts. We evaluated the prediction of obstructive disease from combined analysis of semiupright and supine stress MPI by deep learning (DL) as compared with standard combined total perfusion deficit (TPD). Methods: 1,160 patients without known coronary artery disease (64% male) were studied. Patients underwent stress 99mTc-sestamibi MPI with new-generation solid-state SPECT scanners in 4 different centers. All patients had on-site clinical reads and invasive coronary angiography correlations within 6 mo of MPI. Obstructive disease was defined as at least 70% narrowing of the 3 major coronary arteries and at least 50% for the left main coronary artery. Images were quantified at Cedars-Sinai. The left ventricular myocardium was segmented using standard clinical nuclear cardiology software. The contour placement was verified by an experienced technologist. Combined stress TPD was computed using sex- and camera-specific normal limits. DL was trained using polar distributions of normalized radiotracer counts, hypoperfusion defects, and hypoperfusion severities and was evaluated for prediction of obstructive disease in a novel leave-one-center-out cross-validation procedure equivalent to external validation. During the validation procedure, 4 DL models were trained using data from 3 centers and then evaluated on the 1 center left aside. Predictions for each center were merged to have an overall estimation of the multicenter performance. Results: 718 (62%) patients and 1,272 of 3,480 (37%) arteries had obstructive disease. The area under the receiver operating characteristics curve for prediction of disease on a per-patient and per-vessel basis by DL was higher than for combined TPD (per-patient, 0.81 vs. 0.78; per-vessel, 0.77 vs. 0.73; P < 0.001). With the DL cutoff set to exhibit the same specificity as the standard cutoff for combined TPD, per-patient sensitivity improved from 61.8% (TPD) to 65.6% (DL) (P < 0.05), and per-vessel sensitivity improved from 54.6% (TPD) to 59.1% (DL) (P < 0.01). With the threshold matched to the specificity of a normal clinical read (56.3%), DL had a sensitivity of 84.8%, versus 82.6% for an on-site clinical read (P = 0.3). Conclusion: DL improves automatic interpretation of MPI as compared with current quantitative methods.

SPECT myocardial perfusion imaging (MPI) is widely used for the diagnosis of coronary artery disease (CAD), with more than 6.3 million scans performed annually in North America (1). High-efficiency SPECT scanners equipped with solid-state detectors and specialized collimators have dramatically improved count sensitivity and image resolution (2). They enable shorter acquisition times that facilitate imaging patients in multiple positions to assess image artifacts, or alternatively the implementation of low radiation protocols by performing standard time acquisitions or stress-only protocols (3).

Machine learning process allows artificial intelligence systems to acquire their own knowledge from raw data. Recent research has demonstrated increased capabilities of deep convolutional neural networks (often referred to as deep learning [DL]) to solve challenging tasks such as classification and image segmentation (4). In contrast to conventional machine learning that requires predefined image features to be computed in advance (5,6), convolutional networks are connected directly to image pixels and learn image statistics in a self-taught manner, therefore processing the images in their natural form (4). DL ability to simultaneously analyze normalized radiotracer counts, hypoperfusion defects, and hypoperfusion severities on SPECT-MPI maps to predict CAD has been recently demonstrated (7).

In this work, we aimed to apply DL to upright and supine images acquired for the same patient on a high-efficiency camera. Physicians need to review these 2 images for the final interpretation. Simple rule-based computational methods are currently used for combined quantification of upright/supine images (8) to mitigate various imaging artifacts, such as attenuation artifacts. In this study, using a large international cohort of high-efficiency MPI with correlating invasive coronary angiography (ICA), we apply DL to improve the interpretation of upright/supine images.

MATERIALS AND METHODS

Study Population

The studied dataset was collected under NIH-sponsored REgistry of Fast Myocardial Perfusion Imaging with NExt generation SPECT (REFINE-SPECT) (9). The registry contains MPI studies and on-site clinical reads of consecutive patients without known CAD who underwent clinically indicated ICA within 180 d of MPI (7). This information was transferred to the core laboratory (Cedars-Sinai) for processing. We analyzed stress MPI images from 1,160 patients (64% male) from the registry who underwent upright and supine SPECT MPI (performed on a D-SPECT system) in 4 nuclear cardiology centers in the United States, between 2008 and 2015. The study was approved by the institutional review board of each center, and the requirement to obtain informed consent was waived.

Stress Image Acquisition

99mTc-sestamibi stress imaging was performed using high-efficiency solid-state D-SPECT scanners (Spectrum-Dynamics) for patients positioned semiupright and supine (10). Patients underwent either symptom-limited Bruce protocol treadmill exercise testing only (522 [45%]) or pharmacologic stress (638 [55%]), with radiotracer injection at peak exercise or during maximal hyperemia, respectively. Upright and supine stress imaging began 15–60 min after stress and followed fast acquisition protocols—acquisition lasting 4–6 min each. Reconstructed images were generated on-site from the list-mode data by vendor-recommended iterative reconstruction with resolution recovery as optimized on this scanner (10). No attenuation, scatter, or motion correction was applied.

ICA

ICA was performed within 180 d of the MPI examination according to standard clinical protocols and routine. All coronary angiograms were visually interpreted by an on-site cardiologist, independent from MPI but not formally blinded to MPI. Luminal diameter narrowing of 50% or greater in the left main artery, or of 70% or greater in the left anterior descending (LAD), left circumflex (LCx), or right coronary arteries (RCA), was considered significant and used as the gold standard for obstructive CAD.

Image Processing

Image datasets were transferred and quantified at Cedars-Sinai Medical Center. Left ventricular (LV) myocardial contours were computed using standard Cedars-Sinai Quantitative Perfusion SPECT software, version 2015 (11). LV contours were verified by a technologist with more than 15 y of experience in nuclear cardiology who was blinded to angiographic and clinical findings. When needed, the technologist corrected the gross initial LV localization, the LV mask, and the valve plane position (6,12).

Automated Myocardial Perfusion SPECT Quantification

Polar map samples derived from the raw images by the standard algorithm were used to generate raw polar maps showing radiotracer count distributions normalized to the maximal counts. SPECT images were quantified by sex-specific normal limits deriving upright and supine blackout and total perfusion deficit (TPD) maps—4 polar maps in total (8,13). Blackout polar maps defined areas of hypoperfusion (11) as blacked-out samples in the raw map. TPD polar maps described perfusion deficits—individual hypoperfusion severity—as polar map samples on a 0–4 scale (13).

Standard Measures

Standard clinical combined TPD (cTPD) measures per patient and per vessel for LAD, LCx, and RCA territories were computed using the rule of concomitant location of the perfusion defect (8) for comparison to DL. Final on-site clinical reads incorporating all clinical information and imaging data (static, gated, stress, rest) were reported on a scale of 0–4 (0, normal; 1, probably normal; 2, equivocal; 3, abnormal; 4, definitely abnormal) by experts from each site during clinical reporting. Clinical scores of 0–1 were considered as normal, and 2–4 were considered as abnormal for sensitivity/specificity calculation.

DL

The DL procedure illustrated in Figure 1 extends a previous approach (7) by processing upright and supine polar maps simultaneously. Sex information was included to account for the image differences between men and women on non–attenuation-corrected MPI. The model estimates the obstructive per-vessel CAD probability, without the use of predefined coronary territories or any assumed subdivision of the input polar map. The maximum per-vessel probability of obstructive CAD was retained as the per-patient score.

FIGURE 1.
FIGURE 1.

DL prediction of obstructive CAD from upright and supine MPI. A deep convolutional neural network trained from obstructive stenosis correlations by ICA was used to simultaneously estimate probability of obstructive CAD for LAD, LCx, and RCA territories from upright and supine polar MPI maps. Maximum probability was retained as probability of patient disease. FC = fully connected layer; Max-pooling = function that returns maximum value for image patch; ReLU = rectified linear unit.

Leave-One-Center-Out Cross-Validation

To externally validate the expected performance of the DL model for each participant center, we divided the studied population per center (4 groups) and then trained 4 DL models with data from 3 centers and evaluated them on the remaining external center in a leave-one-center-out cross-validation (Fig. 2) (14). Therefore, the generalizability of the prediction to new MPI data was determined by evaluating unseen patients from a separate center. External validation aims to assess the accuracy of a model in patients from a different but plausibly related population (15). Externally validated predictions for each center were subsequently merged, which reduces the variability of the estimated performance as compared with an external validation in a single external center.

FIGURE 2.
FIGURE 2.

Leave-one-center-out cross-validation. Input stress MPI datasets are divided by center (4 in total). Four folds are built, each containing training sample made up of images from 3 centers and validation sample with images from remaining center. This procedure allows external validation of 4 DL models trained separately in each fold.

Implementation

The DL model was implemented using the convolutional architecture for fast feature embedding DL toolkit (Berkeley Artificial Intelligence Research Lab) in Python programming language 2.7.12 (7,16). Model training was performed on graphics processing units in a GeForce GTX 1080 Ti card (Nvidia).

Statistical Analysis

DL and cTPD were compared using pairwise comparisons of the area under the receiver operating characteristic curve (AUC) according to DeLong et al. (17). Per-vessel curves were computed by merging LAD, LCx, and RCA scores by per-vessel cTPD and by the DL scores. The McNemar test was used to assess differences in sensitivity. The per-patient and per-vessel improvements in sensitivity were computed for the DL thresholds matching the specificity when using previously established diagnostic cutoff values of 3% for per-patient cTPD and 1% for per-vessel cTPD (8). Per-patient sensitivity was also evaluated for DL and cTPD thresholds matching the specificity of clinical readers. A 2-tailed P value of less than 0.05 was considered significant. Statistical calculations were performed in R software, version 3.4 (18).

RESULTS

Baseline characteristics of the studied population (Table 1) were similar to reported single-center studies, with a higher incidence of diabetes (8,19). The prevalence of obstructive CAD (Table 2), age, incidence of diabetes mellitus, and incidence of hypertension (Supplemental Tables 1 and 2) were the same across centers. Injected radiotracer activities for stress image acquisition are in Table 3.

View this table:
TABLE 1

Baseline Characteristics of Studied Population

View this table:
TABLE 2

Prevalence of Obstructive CAD

View this table:
TABLE 3

Radiotracer Activity for Stress Image Acquisition

LV Segmentation

LV contours were manually corrected in 143 (12.3%) upright images and 238 (20.5%) supine images (P < 0.0001). Upright image modifications included 142 corrections for LV location, 15 corrections for LV mask, and 139 valve plane corrections (12). Supine image modifications included 238 for LV location, 29 for LV mask, and 200 valve plane corrections.

DL

Overall, the leave-one-center-out cross-validation training/validation loop took less than 30 min using graphics processing units, which includes the creation of 4 DL models. In the testing phase, the prediction of LAD, LCx, and RCA disease could be completed in less than 0.5 s per patient using central-processing-unit computation.

Per-Patient

The merged per-patient AUC by DL was significantly higher than the AUC by cTPD (Fig. 3A). When operating with the same specificity as cTPD with a per-patient threshold of 3%, DL significantly improved the sensitivity of per-patient disease prediction. When operating with the same specificity as normal clinical reads, DL had the same sensitivity for disease prediction as on-site clinical readers and significantly improved the sensitivity compared with cTPD (Fig. 4).

FIGURE 3.
FIGURE 3.

DL prediction of obstructive CAD from 4 externally validated models with merged data for 4 centers. Per-patient (A) and per-vessel (B) DL predictions of obstructive CAD from upright and supine images (DL, red) are compared with prediction of obstructive CAD by combined upright-supine TPD (cTPD, blue). AUC per center was externally validated using CAD scores from 4 different DL models (1 per center) with each model trained with data from other 3 centers. Red dotted line (bottom) shows overall multicenter AUC. CI = confidence interval; ROC = receiver operating characteristic.

FIGURE 4.
FIGURE 4.

Sensitivity for prediction of obstructive CAD. Per-patient DL prediction of obstructive CAD by DL computed from upright and supine MPI (DL, red) had higher sensitivity than prediction by cTPD (blue) and same sensitivity as on-site clinical readers (green). DL cutoff was set to 0.29, and cTPD cutoff was set to 0.62% to exhibit same specificity as normal or probably-normal clinical read. CI = confidence interval.

Per-Vessel

The merged per-vessel AUC by DL was significantly higher than the AUC by per-vessel cTPD (Fig. 3B). When operating with the same specificity as cTPD with a per-vessel threshold of 1%, DL significantly improved the sensitivity of per-vessel disease prediction.

Per-Center Analysis

DL resulted in significant improvements of per-patient and per-vessel AUCs as compared with cTPD for all externally validated centers except for per-patient CAD prediction in center 3, where they were similar (Fig. 3 [bottom]).

Performance per Subpopulation

DL resulted in significant improvements as compared with cTPD for all studied subpopulations except for per-patient AUC for women and for patients undergoing exercise stress test, for whom DL performed the same (Fig. 5).

FIGURE 5.
FIGURE 5.

Prediction of obstructive CAD per subpopulation. REFINE-SPECT subpopulations were defined by sex (F, M), obesity (nonobese: body mass index < 30 kg/m2, obese: body mass index ≥ 30 kg/m2), stress imaging activity (low: patients undergoing stress-first/stress-only MPI; standard: patients undergoing rest-first/2-d MPI), and stress test type (exercise, pharmacologic). Red dotted line shows overall multicenter AUC as reported in Figure 3. BMI = body mass index; CI = confidence interval.

Case Examples

Figure 6 illustrates 2 cases of obstructive CAD correctly predicted by DL. Figure 6A shows a case of LAD disease correctly predicted by DL but with normal per-vessel and per-patient cTPD. Figure 6B shows a case of triple-vessel disease correctly identified by DL but with normal TPD at the LAD and LCx territories.

FIGURE 6.
FIGURE 6.

Prediction of obstructive CAD from upright and supine stress MPI. Short/long axis views, polar maps depicting normalized radiotracer count distribution and perfusion defects (top), and predictions by cTPD and DL (bottom) are shown for 2 patients with obstructive CAD. (A) In 79-y-old man (85% proximal LAD stenosis) quantified with normal cTPD (per-patient cTPD < 3% and per-vessel cTPD < 1%), DL correctly identified LAD disease. Patient had body mass index of 30 kg/m2 and diabetes and underwent exercise stress MPI. (B) In 62-y-old woman (70% mid LAD stenosis, 95% proximal LCX stenosis, and 80% proximal RCA stenosis) with cTPD abnormal for 1 vessel only, DL correctly identified triple-vessel disease. Patient had body mass index of 25 kg/m2, dyslipidemia, and family history of cardiac disease and underwent exercise stress MPI. BMI = body mass index.

DISCUSSION

In this study, we applied DL to automatically combine upright and supine MPI polar maps and to predict obstructive CAD. The performance of DL was compared to clinically established combined perfusion quantification by upright and supine TPD using the rule of the concomitant location of the defect (8,13), and to on-site clinical readers. D-SPECT MPI data were collected in a large multicenter registry, to our knowledge the largest to date with ICA correlations, with the number of samples per vessel disease similar to the number of samples per category in computer vision datasets or new applications in medicine (7,20,21).

In a novel leave-one-center-out cross-validation procedure, equivalent to external validation, we observed that DL from upright and supine polar MPI images outperformed cTPD in the prediction of obstructive disease. The observed gains for CAD prediction were greater than those obtained by attenuation-corrected TPD, or by visual analysis of attenuation-corrected SPECT MPI images (22). These gains by DL derive from an improved processing of the same information used by cTPD with no additional testing, radiation, or cost. It should also be kept in mind that there is an upper limit in diagnostic MPI performance measured versus ICA due to the physiologic constraint of prediction of stenosis from a perfusion defect.

We demonstrate that DL matched the sensitivity/specificity of disease prediction by clinical expert reading, even though on-site reader experts made their diagnosis with all imaging data (including rest scans and gated scans) as well as patient’s clinical information, all unknown by DL and cTPD. Moreover, experts reported in their own laboratory, whereas DL was trained on data excluding patients from the tested laboratory. This validation approach conservatively reflects the performance estimate of the generalization of the DL approach to unseen centers for potential external deployment of such technology.

The proposed DL procedure performs an automatic end-to-end prediction of CAD (from the image pixels to the predicted CAD score) of upright and supine MPI. This demonstrates DL flexibility and extends the previous single-view approach (7). The DL procedure learned image statistics from supine and upright maps and integrated them with sex information to compute a score for obstructive CAD outperforming cTPD prediction. The DL procedure was able to capture complex relationships that were not easily captured with the rule for upright and supine concomitancy as observed on the clinical examples in Figure 6.

We observed AUC improvements for men and for patients undergoing pharmacologic stress, same-day rest/stress, 2-d stress/rest, or stress-only MPI. We also observed per-patient and per-vessel AUC improvements by DL as compared with cTPD, for each participant center. Some intercenter AUC variability was observed, likely due to the differences in patients between centers. For instance, we observed a higher proportion of patients undergoing the exercise stress test in center 1 than the other 2 centers (Supplemental Table 1; supplemental materials are available at http://jnm.snmjournals.org). The variability may also be due to the differences in imaging protocols for obtaining the upright and supine images among the centers.

The proposed DL method can be easily and immediately deployed clinically. The expected increased performance should be reproduced in new upright/supine data from unseen centers as observed here after rigorous external validation. Potentially, the method can be extended for the simultaneous analysis of prone and supine images or for attenuation-corrected and non–attenuation-corrected images. The proposed method is standalone and uses only image data, combining optimally the upright and supine views, which is difficult visually. In comparison to machine learning models combining image and clinical information (23), the presented technique does not require entering clinical information by the physician, which is potentially dependent on the automatic access to the electronic health records. The predicted per-vessel probabilities of obstructive CAD could be integrated with the quantitative software, in the form of polar maps (Fig. 6) to aid the clinician in final reporting. The execution time of such DL models (<1 s), allows for routine application of what can be the first practical deployment of DL technology to nuclear medicine and cardiac imaging and one of the first examples of DL application in clinical practice.

This study has several limitations. First, visual stenosis on ICA was used as the gold standard, which is known to overestimate the prevalence of functionally significant disease when compared with fractional flow reserve. Fractional flow reserve measurements were not available in this population, as these are not commonly performed clinically. The accuracy of stenosis interpretation may also differ between centers. The established endpoint interval of 180 d between SPECT MPI and clinical indicated ICA may be considered long. We did use a 70% diameter stenosis cutoff for the 3 major coronary arteries to indicate functionally significant lesions (24). Second, in this study, we used upright and supine polar maps acquired in D-SPECT cameras because this protocol is specific to this camera. Motion correction was not considered for these scanners because it was not available at any of the participant centers at the time when the images were collected and interpreted; however, the proposed simultaneous processing of 2-position images could potentially mitigate motion artifacts in 1-position images. Third, rest scans and ischemia were not considered; however, in this work, we studied patients without known CAD, for which, traditionally, stress variables are used for detection of obstructive disease. It is possible that externally validated machine learning models, also incorporating clinical information into the scoring of obstructive disease (5), provide further enhancements in AUC performance, especially for the prediction of prognostic outcomes.

CONCLUSION

DL improves automatic prediction of obstructive CAD from MPI, as compared with the current standard quantitative method.

DISCLOSURE

This research was supported in part by grant R01HL089765 from the National Heart, Lung, and Blood Institute/National Institutes of Health (NHLBI/NIH) (principal investigator, Piotr Slomka). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Daniel Berman, Guido Germano, and Piotr Slomka participate in software royalties for Quantitative Perfusion SPECT (QPS) software at Cedars-Sinai Medical Center. Andrew Einstein has served as a consultant to GE Healthcare and received research support from Philips Healthcare and Toshiba America Medical Systems. Terrence Ruddy serves as a consultant and has received research support from GE Healthcare, Advanced Accelerator Applications, and AstraZeneca. Sharmila Dorbala serves as a consultant to GE Healthcare and has received consulting honoraria from Bracco. No other potential conflict of interest relevant to this article was reported.

Acknowledgments

We want to thank all the people whose efforts allowed us to collect, process, and analyze the data in the NIH-sponsored REFINE-SPECT registry.

Footnotes

  • Published online Sep. 27, 2018.

REFERENCES

  1. 1.↵
    1. Einstein AJ
    . Effects of radiation exposure from cardiac imaging: how good are the data? J Am Coll Cardiol. 2012;59:553–565.
    OpenUrlFREE Full Text
  2. 2.↵
    1. Slomka PJ,
    2. Patton JA,
    3. Berman DS,
    4. Germano G
    . Advances in technical aspects of myocardial perfusion SPECT imaging. J Nucl Cardiol. 2009;16:255–276.
    OpenUrlCrossRefPubMed
  3. 3.↵
    1. Slomka PJ,
    2. Dey D,
    3. Duvall WL,
    4. Henzlova MJ,
    5. Berman DS,
    6. Germano G
    . Advances in nuclear cardiac instrumentation with a view towards reduced radiation exposure. Curr Cardiol Rep. 2012;14:208–216.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. LeCun Y,
    2. Bengio Y,
    3. Hinton G
    . Deep learning. Nature. 2015;521:436–444.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Arsanjani R,
    2. Xu Y,
    3. Dey D,
    4. et al
    . Improved accuracy of myocardial perfusion SPECT for detection of coronary artery disease by machine learning in a large population. J Nucl Cardiol. 2013;20:553–562.
    OpenUrl
  6. 6.↵
    1. Betancur J,
    2. Rubeaux M,
    3. Fuchs T,
    4. et al
    . Automatic valve plane localization in myocardial perfusion SPECT/CT by machine learning: anatomical and clinical validation. J Nucl Med. 2017;58:961–967.
    OpenUrlAbstract/FREE Full Text
  7. 7.↵
    1. Betancur J,
    2. Commandeur F,
    3. Motlagh M,
    4. et al
    . Deep learning for prediction of obstructive disease from fast myocardial perfusion SPECT: a multicenter study. JACC Cardiovasc Imaging. 2018;11:1654–1663.
    OpenUrlAbstract/FREE Full Text
  8. 8.↵
    1. Nakazato R,
    2. Tamarappoo BK,
    3. Kang X,
    4. et al
    . Quantitative upright–supine high-speed SPECT myocardial perfusion imaging for detection of coronary artery disease: correlation with invasive coronary angiography. J Nucl Med. 2010;51:1724–1731.
    OpenUrlAbstract/FREE Full Text
  9. 9.↵
    1. Slomka PJ,
    2. Betancur J,
    3. Liang JX,
    4. et al
    . Rationale and design of the registry of fast myocardial perfusion imaging with next generation SPECT (REFINE SPECT). J Nucl Cardiol. June 19, 2018 [Epub ahead of print].
  10. 10.↵
    1. Gambhir SS,
    2. Berman DS,
    3. Ziffer J,
    4. et al
    . A novel high-sensitivity rapid-acquisition single-photon cardiac imaging camera. J Nucl Med. 2009;50:635–643.
    OpenUrlAbstract/FREE Full Text
  11. 11.↵
    1. Germano G,
    2. Kavanagh PB,
    3. Slomka PJ,
    4. Van Kriekinge SD,
    5. Pollard G,
    6. Berman DS
    . Quantitation in gated perfusion SPECT imaging: the Cedars-Sinai approach. J Nucl Cardiol. 2007;14:433–454.
    OpenUrlCrossRefPubMed
  12. 12.↵
    1. Xu Y,
    2. Kavanagh P,
    3. Fish M,
    4. et al
    . Automated quality control for segmentation of myocardial perfusion SPECT. J Nucl Med. 2009;50:1418–1426.
    OpenUrlAbstract/FREE Full Text
  13. 13.↵
    1. Slomka PJ,
    2. Nishina H,
    3. Berman DS,
    4. et al
    . Automated quantification of myocardial perfusion SPECT using simplified normal limits. J Nucl Cardiol. 2005;12:66–77.
    OpenUrlCrossRefPubMed
  14. 14.↵
    1. Steyerberg EW,
    2. Harrell FE
    . Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol. 2016;69:245–247.
    OpenUrlCrossRefPubMed
  15. 15.↵
    1. Bleeker SE,
    2. Moll HA,
    3. Steyerberg EW,
    4. et al
    . External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003;56:826–832.
    OpenUrlCrossRefPubMed
  16. 16.↵
    1. Jia Y,
    2. Shelhamer E,
    3. Donahue J,
    4. et al
    . Caffe: convolutional architecture for fast feature embedding. arXiv.org website. https://arxiv.org/abs/1408.5093. Published June 20, 2014. Accessed December 19, 2018.
  17. 17.↵
    1. DeLong ER,
    2. DeLong DM,
    3. Clarke-Pearson DL
    . Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–845.
    OpenUrlCrossRefPubMed
  18. 18.↵
    Team RCR. A language and environment for statistical computing. R Foundation for Statistical Computing. 2017.
  19. 19.↵
    1. Nishina H,
    2. Slomka PJ,
    3. Abidov A,
    4. et al
    . Combined supine and prone quantitative myocardial perfusion SPECT: method development and clinical validation in patients with no known coronary artery disease. J Nucl Med. 2006;47:51–58.
    OpenUrlAbstract/FREE Full Text
  20. 20.↵
    1. Deng J,
    2. Dong W,
    3. Socher R,
    4. Li LJ,
    5. Li K,
    6. Fei-Fei L
    . ImageNet: a large-scale hierarchical image database. IEEE Xplore website. https://ieeexplore.ieee.org/document/5206848. Published August 18, 2009. Accessed December 19, 2018.
  21. 21.↵
    1. Yousefi S,
    2. Amrollahi F,
    3. Amgad M,
    4. et al
    . Predicting clinical outcomes from large scale cancer genomic profiles with deep survival models. Sci Rep. 2017;7:11707.
    OpenUrl
  22. 22.↵
    1. Arsanjani R,
    2. Xu Y,
    3. Hayes SW,
    4. et al
    . Comparison of fully automated computer analysis and visual scoring for detection of coronary artery disease from myocardial perfusion SPECT in a large population. J Nucl Med. 2013;54:221–228.
    OpenUrlAbstract/FREE Full Text
  23. 23.↵
    1. Betancur J,
    2. Otaki Y,
    3. Motwani M,
    4. et al
    . Prognostic value of combined clinical and myocardial perfusion imaging data using machine learning. JACC Cardiovasc Imaging. 2018;11:1000–1009.
    OpenUrlAbstract/FREE Full Text
  24. 24.↵
    1. Tonino PAL,
    2. Fearon WF,
    3. De Bruyne B,
    4. et al
    . Angiographic versus functional severity of coronary artery stenoses in the FAME study: fractional flow reserve versus angiography in multivessel evaluation. J Am Coll Cardiol. 2010;55:2816–2821.
    OpenUrlFREE Full Text
  • Received for publication April 23, 2018.
  • Accepted for publication September 14, 2018.
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Bookmark this article

Jump to section

Related Articles

Cited By...

More in this TOC Section

Similar Articles

Keywords