Comparison of interpretability methods in the context of deep neural networks for radiomics application

Introduction: Deep neural networks have shown potential in Radiomics application to facilitate and automate the development of diagnostic, prognostic and predictive models exploiting large amounts of imaging data. However they are sometimes considered as untrustworthy black boxes by end-users. Any clinical decision-aid system built using these new methods should be as explainable and interpretable as possible to improve trust, e.g. by yielding understandable explanations regarding their predictions. Existing interpretability methods can help provide such information, including through visual feedback highlighting the most decisive elements of images, without the need for advanced knowledge regarding neural networks. The goal of this work was to compare the relevance of several such methods in the context of convolutional neural networks trained to classify medical images.

Methods: The evaluated interpretability methods in this work are all gradient-based, using Backpropagation as the foundation of their interpretation process: Gradient Saliency (GS), Integrated Gradients (IG) and Expected Gradients (EG). The output of these methods are attribution maps quantifying the influence of all input image pixels on the network’s prediction. In addition, we integrated these 3 methods within the recently proposed XRAI approach, thereby creating region-wise attribution maps (Figure 1), and giving the ability to display only the most influential regions of the input image in terms of percentage. The comparison was first carried out in an application where performance is expected to be very high and where the human interpretation can be easily understood: detection of contrast enhancement agent in chest computed tomography (CT) scans. A ResNet-50 network taking as input a single slice, and outputting a single value (with or without contrast enhancement) was trained on both LIDC-IDRI and LNDb databases, for a total of 1312 patients scans containing ~320000 slices. In order to quantitatively evaluate the ability of interpretability methods to highlight the most relevant regions for prediction, ground-truth masks were defined from clinician expert knowledge (mostly the heart and aorta regions) in 30 CT-slices from as many patients. Relevant regions from the 3 different methods were compared with the ground-truth reference through precision, recall and F1 score. Generating a consensus of the 3 approaches at different steps of the whole process was also attempted.

Results: After training, the network classified images with 99.7% accuracy. Pixel-based attribution maps are difficult to interpret visually, compared to the ones obtained through the XRAI process (Figure 1). According to the quantitative evaluation, EG led to the highest overlap between the ground-truth and the highlighted regions, thus seems to be the most accurate for providing relevant interpretability for this simple network, according to chosen metrics (Figure 2). Generating a consensus of the three methods did not improve the results compared to EG.

Conclusions: Our comparison in this preliminary investigation supports the ability of gradient-based methods (especially EG) to provide relevant visual feedback to users for neural network’s prediction. Our work is currently limited to exploiting a single family of interpretability methods (gradient-based) and other methods will be explored in the future, which might also be more interesting in the context of generating a consensus of several different approaches. Another limitation is the use of a very simple “toy example” application, although this was done on purpose to facilitate the quantitative evaluation with respect to a specific reference generated by a human expert. Additional experiments on the same datasets but different tasks (ex. classify images according to the scanner model slice thickness) as well as different datasets and applications (predicting outcome in PET/CT cohorts) are currently underway and more results will be presented at the conference.

• Download figure
• Open in new tab
• Download powerpoint

• Download figure
• Open in new tab
• Download powerpoint