Skip to main content

Main menu

  • Home
  • Content
    • Current
    • Ahead of print
    • Past Issues
    • JNM Supplement
    • SNMMI Annual Meeting Abstracts
    • Continuing Education
    • JNM Podcasts
  • Subscriptions
    • Subscribers
    • Institutional and Non-member
    • Rates
    • Journal Claims
    • Corporate & Special Sales
  • Authors
    • Submit to JNM
    • Information for Authors
    • Assignment of Copyright
    • AQARA requirements
  • Info
    • Reviewers
    • Permissions
    • Advertisers
  • About
    • About Us
    • Editorial Board
    • Contact Information
  • More
    • Alerts
    • Feedback
    • Help
    • SNMMI Journals
  • SNMMI
    • JNM
    • JNMT
    • SNMMI Journals
    • SNMMI

User menu

  • Subscribe
  • My alerts
  • Log in
  • My Cart

Search

  • Advanced search
Journal of Nuclear Medicine
  • SNMMI
    • JNM
    • JNMT
    • SNMMI Journals
    • SNMMI
  • Subscribe
  • My alerts
  • Log in
  • My Cart
Journal of Nuclear Medicine

Advanced Search

  • Home
  • Content
    • Current
    • Ahead of print
    • Past Issues
    • JNM Supplement
    • SNMMI Annual Meeting Abstracts
    • Continuing Education
    • JNM Podcasts
  • Subscriptions
    • Subscribers
    • Institutional and Non-member
    • Rates
    • Journal Claims
    • Corporate & Special Sales
  • Authors
    • Submit to JNM
    • Information for Authors
    • Assignment of Copyright
    • AQARA requirements
  • Info
    • Reviewers
    • Permissions
    • Advertisers
  • About
    • About Us
    • Editorial Board
    • Contact Information
  • More
    • Alerts
    • Feedback
    • Help
    • SNMMI Journals
  • View or Listen to JNM Podcast
  • Visit JNM on Facebook
  • Join JNM on LinkedIn
  • Follow JNM on Twitter
  • Subscribe to our RSS feeds
LetterLetters to the Editor

Reply: A Common Mistake in Assessing the Diagnostic Value of a Test: Failure to Account for Statistical and Methodologic Issues

Aseem U. Anand and David Minarik
Journal of Nuclear Medicine July 2017, 58 (7) 1183; DOI: https://doi.org/10.2967/jnumed.117.190645
Aseem U. Anand
*Lund University Waldenströms Gata 5, Second Floor Malmö 20502, Sweden E-mail:
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: aseem.anand@med.lu.se
David Minarik
*Lund University Waldenströms Gata 5, Second Floor Malmö 20502, Sweden E-mail:
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: aseem.anand@med.lu.se
  • Article
  • Info & Metrics
  • PDF
Loading

REPLY: Thank you for your interest and comments regarding our work (1). We agree that reproducibility/reliability and accuracy/validity are two different methodologic issues and should be evaluated using appropriate tests for each. Further, we also concur that for quantitative variables, the intraclass correlation coefficient should be used to test reliability and the interclass correlation coefficient (Pearson r) should be used to test validity. Since the automated bone scan index (BSI) is a quantitative parameter, the comments on qualitative assessments are inapplicable.

Our analytic validation study demonstrated the validity of the automated BSI with a Pearson r of 0.99 (P < 0.0001) and with associated parameters of linear regression (slope, 0.80 [95% confidence interval, 0.78–0.83]; intercept, 0.38 [95% confidence interval, 0.25–0.51]) (2). In this study, Cohen κ-agreement and other well-known standard tests for imaging biomarkers (3) were also used to evaluate the reliability of the automated BSI platform. These tests demonstrated a coefficient of variation (<20%) with a consistent linearity—satisfying the homoscedasticity and Shapiro–Wilken test—from low-burden to high-burden disease. The Bland–Altman plot of absolute difference in BSI readings of repeated scans revealed the analytic noise with an SD of 0.15. The study concluded that in the clinical setting, the assessment of change in BSI above the noise threshold (2 SDs, or 0.30) had high interobserver agreement (Cohen κ = 0.96).

Despite the analytic validation of BSI, variation in the imaging procedure can significantly affect the analytic performance of the imaging biomarker. Therefore, in a subsequent publication, we analyzed the effect of variability in scanning speed and γ-camera settings on the BSI assessment (1). The objective of our preanalytic study was not to evaluate the performance characteristics of BSI but to use appropriate statistical tests to evaluate the effect of procedural variability on the analytically validated BSI values. The Wilcoxon/Mann–Whitney test (4) demonstrated that the noise in the BSI value was significantly higher because of variability in scanning speed in comparison with the known analytic gold standard (accuracy performance) and with repeated measurement of test–retest patient bone scans (reproducibility performance). The P value in our study, from the Wilcoxon/Mann–Whitney test, did not imply clinical significance but referred to the statistical significance of the degree of systematic difference between the two BSI readings of the repeated bone scans that were performed in a specific study design to assess the effect of procedural variability on the known performance characteristics of BSI.

In conclusion, the statistical methodology must be reviewed in the context of the study design and its objective. We have performed a comprehensive assessment to demonstrate the reliability and validity of BSI, and with the preanalytic study, we demonstrated that the BSI performance characteristics were dependent on scanning speed. The study added empiric evidence toward the standardization of bone scan acquisitions for robust quantitative BSI assessment in multiinstitutional studies. Together, the analytic and preanalytic studies served as the foundation for prospective clinical investigations aimed at validating automated BSI as a quantitative imaging biomarker.

Footnotes

  • Published online Apr. 6, 2017.

  • © 2017 by the Society of Nuclear Medicine and Molecular Imaging.

REFERENCES

  1. 1.↵
    1. Anand A,
    2. Morris MJ,
    3. Kaboteh R,
    4. et al
    . A preanalytic validation study of automated bone scan index: effect on accuracy and reproducibility due to the procedural variabilities in bone scan image acquisition. J Nucl Med. 2016;57:1865–1871.
    OpenUrlAbstract/FREE Full Text
  2. 2.↵
    1. Anand A,
    2. Morris MJ,
    3. Kaboteh R,
    4. et al
    . Analytic validation of the automated bone scan index as an imaging biomarker to standardize quantitative changes in bone scans of patients with metastatic prostate cancer. J Nucl Med. 2016;57:41–45.
    OpenUrlAbstract/FREE Full Text
  3. 3.↵
    1. O’Connor JP,
    2. Aboagye EO,
    3. Adams JE,
    4. et al
    . Imaging biomarker roadmap for cancer studies. Nat Rev Clin Oncol. 2017;14:169–186.
    OpenUrl
  4. 4.↵
    1. Szklo M,
    2. Nieto FJ
    . Epidemiology Beyond the Basics. 2nd ed. Manhattan, NY: Jones and Bartlett; 2007.
PreviousNext
Back to top

In this issue

Journal of Nuclear Medicine: 58 (7)
Journal of Nuclear Medicine
Vol. 58, Issue 7
July 1, 2017
  • Table of Contents
  • Table of Contents (PDF)
  • About the Cover
  • Index by author
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Journal of Nuclear Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Reply: A Common Mistake in Assessing the Diagnostic Value of a Test: Failure to Account for Statistical and Methodologic Issues
(Your Name) has sent you a message from Journal of Nuclear Medicine
(Your Name) thought you would like to see the Journal of Nuclear Medicine web site.
Citation Tools
Reply: A Common Mistake in Assessing the Diagnostic Value of a Test: Failure to Account for Statistical and Methodologic Issues
Aseem U. Anand, David Minarik
Journal of Nuclear Medicine Jul 2017, 58 (7) 1183; DOI: 10.2967/jnumed.117.190645

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Reply: A Common Mistake in Assessing the Diagnostic Value of a Test: Failure to Account for Statistical and Methodologic Issues
Aseem U. Anand, David Minarik
Journal of Nuclear Medicine Jul 2017, 58 (7) 1183; DOI: 10.2967/jnumed.117.190645
Twitter logo Facebook logo LinkedIn logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One
Bookmark this article

Jump to section

  • Article
    • Footnotes
    • REFERENCES
  • Info & Metrics
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Reply to “The Randomized, Phase 2 LuCAP Study”
  • Business Model Beats Science and Logic: Dosimetry and Paucity of Its Use
  • Patient-Specific Dosimetry-Driven PRRT: Time to Move Forward!
Show more Letters to the Editor

Similar Articles

SNMMI

© 2025 SNMMI

Powered by HighWire