TO THE EDITOR:
We read with interest two back-to-back companion articles written in The Journal of Nuclear Medicine (JNM) by Germano et al. (1) and Sharir et al. (2). In these articles the authors describe and validate a new algorithm for quantification of myocardial perfusion. Although this algorithm has many similarities to the CEqual program developed by us (use of normal databases, criteria for abnormality, polar maps, 3fX maps, etc.), those authors have incorporated some differences (e.g., ellipsoid fitting and sampling). They conclude that, “compared to previous methods,” these differences resulted in “substantially higher specificity for the detection and localization of CAD [coronary artery disease], with comparably high sensitivity” (2).
Their conclusion is not substantiated by the data presented in either of these two articles. They arrived at their conclusion by comparing their results to those obtained by Van Train et al. (3). It is incorrect to make their claim based on this comparison for three reasons. First, Bayes theorem tells us that the accuracy of a test is dependent on the prevalence of disease in the population. Only by comparing the two tests in the same population and obtaining significantly better results for one than the other can one test be demonstrated to be superior. Second, the population used by Sharir et al. (2) was made up of patients acquired from and processed at their institution, whereas the population used by Van Train et al. (3) was from a multicenter trial. Results from in-house validations of data from the same center that used the same population to develop its techniques are expected to have higher accuracy than results from multicenter trials. Third, the claim that a new test has a higher specificity than an old test goes against the expectation that, as a result of post-test referral bias, specificity will continue to drop as referring physicians gain trust in a new test and stop sending patients with normal scans to catheterization. If the “normal” patients are sent for a nuclear perfusion study after catheterization, it must be because some borderline lesions were found. It would be informative to know if the 94 patients in this prospective validation were selected consecutively or whether a different selection scheme was used.
Just like many of the authors of these two articles, we also receive royalties from the sale of our quantitative software. The JNM editors should be aware that claims of superiority of one test over another have significant financial implications for both parties. These claims should be allowed only when comparisons are made between two tests using the same patient population (preferably patients from outside the center that developed the technique) and in which investigators have no financial interest.
REFERENCES
REPLY:
Thank you for the opportunity to clarify some points made in our original articles (1,2). First, we would like to stress that CEqual was jointly developed at Cedars-Sinai Medical Center and Emory University by the authors of the original letter to the editor (Van Train and Garcia), two of the authors of this reply, and other investigators currently at other institutions. All of these individuals have a financial interest in CEqual and no incentive to unfairly present its limitations.
As a consequence of pioneering quantitative efforts like CEqual and the concurrent growth in computing power of the workstations used in nuclear medicine, the 1990s saw the development of more computationally demanding algorithms, operating in three-dimensional space and based on sampling schemes not directly related to circumferential profiles and short-axis slices. Although some of the results are still displayed as polar maps, the maps are now conceived as a collection of a constant number of myocardial samples, not several circumferential profiles dependent on heart size. As a result of this approach, comparison with normal limits is now easier and more straightforward. Quantitative gated perfusion SPECT (QPS) represents Cedars-Sinai’s contribution to this new generation of algorithms, which includes techniques developed by several other groups (3).
Van Train and Garcia are correct in stating that direct comparison of different algorithms applied to the same patient population is highly desirable. As we reported previously (4), we performed exactly such a comparison between QPS and CEqual, studying 62 SPECT patients who also underwent coronary angiography. By analysis of receiver operating characteristics, it was found that the basic difference in the performance of the two algorithms lies in the optimal threshold for abnormality, which is the minimal extent of perfusion abnormality required to call a study abnormal. The specificity of QPS and CEqual in those same patients was 73% and 55%, respectively. Another three-dimensional perfusion quantification algorithm has also been reported to result in a higher normalcy rate (58% vs. 34%) than that of CEqual (5). For clarification, the 94 patients referred to in the letter were a consecutive group undergoing perfusion SPECT and angiography.
How is the improved specificity explained? Balancing the de-creasing specificity of nuclear techniques resulting from post-test referral bias is an improvement in specificity achieved by more complete tests, better algorithms, and more accurate quantification. For example, both attenuation correction and gating of a perfusion SPECT study have been clearly reported to result in higher specificity for the detection of coronary artery disease. Studies evaluating the diagnostic accuracy of visual interpretation of stress 99mTc sestamibi have consistently demonstrated specificity of ∼80%, which is comparable with the results we reported for the QPS algorithm (2). Thus, one might argue that it is entirely reasonable to expect algorithms based in the three-dimensional space to better measure perfusion in the three-dimensional myocardium compared with slice-based techniques.
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.