Free-response methodology: alternate analysis and a new observer-performance experiment

Radiology. 1990 Mar;174(3 Pt 1):873-81. doi: 10.1148/radiology.174.3.2305073.

Abstract

The authors review the recently introduced method for analyzing free-response receiver operating characteristic (FROC) observer-performance experiments. These experiments allow multiple abnormalities and observer responses per image and require correct localization for true-positive events. A fundamental plot, the alternative FROC (AFROC), is emphasized, the area under which is a natural index of performance for measuring FROC observer performance. The authors show that it is possible to analyze free-response data in more general ways. The first method, AFROC analysis, involves scoring FROC images in a different way. The second method involves a new observer-performance experiment, the free-response forced error (FFE) experiment. FROC and AFROC analysis was applied to several FROC data sets, and simultaneous FROC-AFROC-FFE experiments were performed with several radiologist observers to test the three methods. The methods were found to be in good agreement with one another.

MeSH terms

  • Humans
  • Observer Variation*
  • ROC Curve*
  • Radiography*
  • Software