Original Article
Various randomized designs can be used to evaluate medical tests

https://doi.org/10.1016/j.jclinepi.2008.06.017Get rights and content

Abstract

Objective

To explore designs for evaluating the prognostic and predictive value of medical tests and their effect on patient outcome.

Study design

Theoretical analysis with examples from the medical literature.

Results

For evaluating the prognostic value of a test, one can include the test at baseline in prognostic studies. To evaluate the value of test in predicting treatment outcome, the test results can be used as baseline information in randomized controlled trials of treatment. To compare the prognostic or predictive value of two or more tests, the test result combinations can be used as baseline information. To evaluate the effect on patient outcome, randomized controlled trials of test strategies are an option. Randomization can apply to all tested or be restricted to specific subgroups, such as those with discordant test results, to increase the efficiency of trials.

Conclusion

The prognostic and predictive value of medical tests can and should be evaluated, to demonstrate the test's ability to guide clinical decision making and to improve patient outcome. Various randomized designs can be used to evaluate the effects on testing on patient outcome.

Introduction

For scientific purposes, it is worth knowing whether or not a result from a medical test corresponds to the truth. Can the test result be trusted? Does a positive result truly reflect presence of the target disease? These are the first questions that come to the mind in the evaluation of medical tests. Yet, from a patient perspective, mere knowledge about the present true state of things is not enough. Patients with complaints want to get better and those without want to maintain their health.

There are multiple ways in which medical tests can affect patients' health. In addition to the direct effects from the diagnostic procedure, there is the information generated by the test. Providing information on the likely cause or other aspects of one's health problems can have both a positive or a negative effect, albeit usually a limited one [1]. Patients want to be informed about the origin of their complaints, even in the absence of a cure. Such information may enable them to find better ways of coping with the complaints, for example, by developing strategies to limit the disabling impact on their daily activities. Yet, the main effects of medical tests on patient outcome will be the result of subsequent clinical decisions guided by the results of the test. Test results can lead to additional testing, or to starting, modifying, or withholding therapeutic interventions.

In many cases then, it is not only just the present health state that is of interest but also the future course of disease. It then follows that the value of information from using medical tests lies not only in the past (where these complaints come from) or the present (do they correspond to disease), but also in the future. The relevance of diagnostic information is closely related to prognosis and depends on associations between test results and future health and between testing and the outcome of treatment.

In this article, we discuss different methods for evaluating the prognostic value of tests. We start from an evaluation of the prognostic accuracy of tests, and then move on to the ability of tests to predict the outcome of treatment. Section 2 with a presentation of randomized designs for evaluating test–treatment combinations and their effect on patient outcome. Section 3 contains an elaboration of these methods for comparing and evaluating multiple tests.

Section snippets

Evaluating a single test

Several studies have examined the need for duplex ultrasonography (US) in patients with a cervical bruit without further symptoms of cerebrovascular disease. To answer this question, an assessment has to be made of the value of duplex US as a test. Such an evaluation may want to look at the amount of agreement between the index test and the clinical reference standard, the best available method to identify the true condition of the carotid arteries. In this case, the reference test will most

Comparing tests and test strategies

In many clinical situations, there are multiple tests available to examine the presence of the target condition. When one wants to compare 2 competing tests, the designs introduced earlier for the evaluation of a single test have to be adapted slightly.

Conclusions

The examples of published randomized diagnostic trials in this article show that it is feasible to perform thorough evaluations of tests in medicine. Other examples of such trials addressed the evaluation of different diagnostic techniques for ventilation-assisted pneumonia, the comparison of multi-detector row CT with digital subtraction angiography, and the use of transcutaneous oxygen and toe pressure to detect critical limb ischemia [22], [23], [24]. Additional examples can be found in

References (32)

  • R.F. Lewis et al.

    Predictive power of duplex ultrasonography in asymptomatic carotid disease

    Ann Intern Med

    (1997)
  • M.N. Babapulle et al.

    Utility of routine exercise treadmill testing early after percutaneous coronary intervention

    BMC Cardiovasc Disord

    (2007)
  • D.J. Sargent et al.

    Clinical trial designs for predictive marker validation in cancer treatment trials

    J Clin Oncol

    (2005)
  • H.P. Adams et al.

    Antithrombotic treatment of ischemic stroke among patients with occlusion or severe stenosis of the internal carotid artery: a report of the Trial of Org 10172 in Acute Stroke Treatment (TOAST)

    Neurology

    (1999)
  • Low molecular weight heparinoid, ORG 10172 (danaparoid), and outcome after acute ischemic stroke: a randomized controlled trial. The Publications Committee for the Trial of ORG 10172 in Acute Stroke Treatment (TOAST) Investigators

    JAMA

    (1998)
  • R. Coté et al.

    Lack of effect of aspirin in asymptomatic patients with carotid bruits and substantial carotid narrowing. The Asymptomatic Cervical Bruit Study Group

    Ann Intern Med

    (1995)
  • Cited by (64)

    • Benefit assessment of diagnostics – Dealing with incomplete evidence

      2023, Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen
    • Pressure ulcer/injury classification today: An international perspective

      2020, Journal of Tissue Viability
      Citation Excerpt :

      From an evidence-based practice perspective accuracy and relative and absolute measurement errors are important characteristics of PU/PI classification, but they do not indicate clinical effectiveness [9,50,56]. Interventional research designs may be needed to evaluate the clinical value of a measurement procedure such as PU/PI classification [57,58]. Depending on the geographic region, health care setting and administrative requirements, different PU/PI classification systems are, and will continue to be used.

    • Effect of Xpert MTB/RIF on clinical outcomes in routine care settings: individual patient data meta-analysis

      2019, The Lancet Global Health
      Citation Excerpt :

      As in randomised controlled trials (RCTs) of tuberculosis therapeutics,10 surrogate markers for patient-important outcomes for diagnostic-test evaluation—such as test accuracy, turnaround time, time to diagnosis, and time to treatment—might not translate onward to clinical outcomes and could be subject to substantial heterogeneity.11 Although therapeutic outcomes are similarly of primary interest in diagnostic RCTs,12,13 they have received less methodological attention than tuberculosis therapeutic trials.14 Five diagnostic RCTs15–19 across 41 sites in South Africa, Brazil, Zimbabwe, Zambia, and Tanzania, assessing the effect of Xpert among people tested for tuberculosis in routine-care settings have been done.

    • Issues and Challenges in the Systematic Evaluation of Biomarker Tests

      2018, Precision Medicine: Tools and Quantitative Approaches
    View all citing articles on Scopus

    Adapted from: Lijmer JG, Bossuyt PMM. Diagnostic Testing and Prognosis: The randomized controlled trial in test evaluation research. In: Knottnerus JA, (Ed) The Evidence Base of Clinical Diagnosis. London: BMJ Books, 2001.

    View full text