An original voxel-wise supervised analysis of tumors with multimodal radiomics to highlight predictive biological patterns

Objectives: Translational applications of predictive and prognostic image-based learning models are challenging due to their lack of interpretability. When using deep learning, Class Activation Maps (CAM) give information about the regions driving the models. Yet, due to the high-level abstraction of deep features, deep CAM are difficult to interpret. We propose a method that combines the interpretability of handcrafted radiomics with a voxel-wise analysis and facilitates the biological interpretation of models.

Methods: A public dataset of 51 soft tissue sarcoma patients including FDG PET and CT scans, and clinical and follow-up information was used [1]. The task was to predict the occurrence of lung metastases within two years after the diagnosis (19 patients with metastases and 32 without). PET and CT images were pre-processed according to IBSI guidelines [2]. A total of 154 radiomic features per voxel were computed with Pyradiomics [3] by sliding a 9³-voxels kernel to mimic the mean local receptive field of a weakly-supervised U-net convolutional neural network (CNN) with 2 max-pooling layers [4]. An “ROI-Global-Average-Pooling” was performed within each tumor region to yield one global 154-features vector per patient. To address the multicollinearity problem in supervised machine learning, global features were then selected by iteratively removing the one with the highest variance inflation factor (VIF) [5] until the maximum VIF in the feature set was <10. Regularized logistic regression with wrapped forward feature selection and weighted loss was used to model the probability of lung metastasis occurrence. Hyper-parameters tuning was performed using 200*5 folds repeated cross-validation. The performance of the model was characterized based on out of bag (OOB) samples using 1000 bootstrap drawings and was compared to those obtained with anatomical tumor volume (ATV), SUVmax, metabolic tumor volume (MTV) and total lesion glycolysis (TLG). For interpretability, bagging of all 1000 models was used to obtain the final radiomic model M1. To avoid data leakage, the data from each patient were analyzed using a sub-bagging model obtained from the aggregation of all models in which the patient data were not used for training. Radiomics CAM were computed by backpropagating the global decision function of the model M1 at the voxel level. Based on the interpretation of the radiomics CAM, we created simple and radiologically interpretable new features. We finally built a “radiological” model M2 following the same machine learning process but using only these new features combined with ATV, SUVmax, MTV and TLG.

Results: Six features were selected through the cross-validation procedure to design the radiomic model M1. The mean OOB ROC AUC of model M1 was 0.85±0.09, to be compared to 0.80±0.09 for SUVmax, 0.73±0.11 for TLG, 0.69±0.11 for ATV, and 0.60±0.12 for MTV. The radiomics CAM highlighted 3 biological patterns associated with the risk of metastasis: substantial and homogeneous non-metabolic tumor region, localized high FDG uptake sub-regions, and some hypodense sub-regions on CT. These findings are concordant with the soft tissue sarcoma grading system based on the biopsy [6]. The radiomics CAM associated with M1 made it possible to define a new feature reflecting the volume of the tumor sub-region devoid of metabolic activity (SUV<40%SUVmax) or with reduced Hounsfield units (<20HU). A 2-features model M2 combining that feature with SUVmax yielded a mean OOB ROC AUC of 0.83±0.09.

Conclusions: We describe a method based on locally-calculated handcrafted radiomic features that highlights the sub-regions and biological signal driving the model prediction. In a situation where the number of data is limited, we demonstrate how that method makes it possible to spatially and quantitatively interpret radiomic models and design simple and robust biomarkers amenable to a biological interpretation for patient stratification.

• Download figure
• Open in new tab
• Download powerpoint