The T.R.U.E. Checklist for Identifying Impactful Artificial Intelligence–Based Findings in Nuclear Medicine: Is It True? Is It Reproducible? Is It Useful? Is It Explainable?

IS IT TRUE?

The question of truth is highly relevant because a large proportion of AI-based studies in medical imaging are still biased by issues well known to data scientists, such as bias in the training population (e.g., sex, ethnicity, and age), data leakage (i.e., test data used explicitly or implicitly during the training phase) (4), or overfitting. This bias most often results in a lack of generalizability of the AI-based model, meaning that the results and reported level of performance will not hold on different datasets (5). By default, we should assume that the findings, especially when outstanding, are biased, and we should chase potential confounding factors by all means. Control experiments (similar to experiments using a sham group or placebo arm in clinical trials) should be used and reported whenever relevant, giving enough evidence that the findings are scientifically valid. For instance, the probability of false-positive findings can be determined by repeating the extensive model-building and model-evaluation process after randomly permuting the label associated with each patient. Expert data scientists should be called on to assist in the possible identification of bias or sources of data leakage, given that these can be subtle and difficult to detect. Medical experts of course remain essential to detect bias or possible confounding effects associated with the composition of the patient samples.

IS IT REPRODUCIBLE?

The reproducibility crisis affects many fields and has been extensively studied and debated (6), including in the field of radiology (7,8). There have been laudable efforts over the last few years to increase transparency, with the very positive trend of data and models being shared more frequently, resulting in an overall improvement in radiomic and AI-based imaging study quality (9). Yet, even when authors share their models developed within well-known frameworks (e.g., TensorFlow or CAFFE [Convolutional Architecture for Fast Feature Embedding]) using one of the many resource-sharing platforms (e.g., GitHub, SourceForge, GitLab, GitKraken, or Bitbucket), this sharing is often not sufficient to actually reproduce the findings, even when the data are also provided. One of the reasons is that most AI-based models are complex and involve many steps and parameters, such as those relating to image preprocessing, data augmentation, and learning schemes, and these are usually not fully described, despite significantly impacting the results. In AI, “the devil is in the details,” as the saying goes. To overcome this reproducibility challenge and move the field forward, we strongly encourage authors to carefully describe their methods and provide data or code (either source code or executable code) that might be needed to reproduce the investigation or test the model on independent data. In addition, similar to the current practice of calling on statistical expertise to validate the statistical methodology used in scientific manuscripts, we recommend calling on specific expertise to practically check that the provided description or material makes it possible to reproduce the findings and test the models on external data. This extra workload on the reviewers would hugely increase the value of published AI-based contributions. We expect contributions that report reproducible methods to have a much greater impact than those that do not.

IS IT USEFUL?

The usefulness should be appreciated with respect to state-of-the-art knowledge and methods, and a comparison of results with previously published data is a good way to assess the usefulness of new findings. Such comparisons can be difficult when different methods are not assessed on the same dataset, because of many possible confounding factors. Sharing of datasets, which can then be used as benchmarks to compare different methods, as in medical imaging challenges (10), can facilitate fair comparison. In what respect the new findings are superior to existing, and often simpler, methods should always be demonstrated. Performance analysis should include metrics characterizing the robustness of the method with respect to potential perturbations (e.g., data of different quality) so as to properly assess the trade-off between complexity, accuracy, and robustness achieved by different models. Occam’s razor should remain the rule until well-supported evidence of the superiority of less intuitive and more complex models is obtained. Although AI is extremely powerful, its power should rather be used when conventional statistical approaches or signal-processing methods are insufficient. There can be different motivations for using an AI model: an AI-based method can save time while equaling human observer performance (11), it can equal human observer performance while reducing interobserver reproducibility (12), it might outperform existing human-based performance (13) or algorithm-based performance (14) (although this will have to be proven in prospective studies), or it might even uncover unknown phenomena (15). Whatever the scenario, the added value of the AI model should be well substantiated.

IS IT EXPLAINABLE?

AI is not a magic wand. It is a powerful set of algorithms that learn from examples and have the unique ability to identify structures in highly dimensional data. When AI is used to automate a task that humans can do by learning from many examples, the rules are deduced by the AI from the examples. The performance to be expected will depend on the representativity of the learning ensemble compared with the cases that can be encountered in practice. A more challenging application is having the AI succeed in doing something that we, as humans, cannot do (yet). As an example, we are currently at a loss when predicting why certain patients will respond to immunotherapy whereas others will not. For these applications, investigating what makes an AI algorithm successful is essential to avoid misinterpretation and prevent overestimation of the power of AI. For example, a misinterpretation of an AI decision-making process was published in a highly respected journal (16), before a reanalysis of the data elegantly demonstrated the incorrect understanding of the initial results (17). This error emphasizes the need for scrutiny of the key elements explaining the performance of an AI-based model. By better understanding the AI model and which specific information it uses, we might also gain knowledge of the biologic mechanisms involved. For this explanation step, speculation is still currently the rule. To use AI as a datascope that will help us better understand the molecular mechanisms based on image content, we have to go from speculation to hypothesis formulation and then hypothesis testing using appropriate in silico, in vitro, or in vivo experimental design.

Explainable AI is currently an extremely active area of research, with the ongoing development of numerous methods for approaching explainability (18), although fully satisfactory explanations may not always be feasible because of the high complexity and dimensionality of the data (19). The “Is it explainable?” question is thus certainly the most difficult one to answer convincingly. Yet, it should not be avoided and should be addressed whenever possible so that AI can help us learn from the data.