Exploring the impact of feature selection methods and classification algorithms on the predictive performance of PET radiomic ML models in lung cancer

Introduction: The high-dimensional data from the radiomics technique can lead to overfitting and poor performance in predicting lung cancer mutation status. Thus, finding the best combination of feature selection and machine learning (ML) methods is crucial to overcome this challenge. The behavior of the ML algorithm can be influenced by features, and the feature selection technique might affect the model performance. The selection of relevant features and removal of unnecessary and redundant features are essential in feature selection to overcome weaknesses of earlier techniques. In this study we explore the use of PET radiomics features, when combined with various feature selection methods and multiple ML classifiers, to evaluate the performance for predicting the mutation status in lung cancer.

Methods: The retrospective study analyzed data from 100 NSCLC patients with available EGFR mutation status as 43 mutant and 57 wild types and corresponding PET/CT data. Tumor segmentation was done with the 3D tool to generate volume of interest based on a 40% threshold of SUVmax on PET data. A total of 111 IBSI complaint PET radiomic features were extracted. Data was preprocessed and normalized to z-score before splitting into training and validation sets in the ratio of 7: 3 respectively. In total 8 feature selection methods were applied including; variance threshold(VT), select k best(SKB), select percentile(SP), stepforward selection(SFS), random forest (RF), recursive feature elimination(RFE), logistic regression with lasso (L1) and, logistic regression with ridge (L2). 7 classifiers were employed including; random forest classifier (RFC), decision tree (DT), k nearest neighbor(KNN), gaussian naïve bayes (GNB), stochastic gradient descent(SGD), quadratic discriminant analysis classifier(QDA), and support vector machine(SVM), to build a total of 56 selection/classifier combinations. Gradient search CV was used for model hyper tuning and 10-fold cross-validation was performed. Performance metrics included cross-validation score, accuracy, precision, and area under the curve (AUC). Radiomic features were extracted from Lifex 7.3.24, whereas for building ML models conda 23.3.1 and python 3.10.9 was used. Libraries for feature selection, classifiers, and metrics were imported from sci-kit-learn 1.3.2.

Results: The number of features selected varied depending on the method, features selected by VT, SKB, SP, SFS, RF, RFE, L1, and L2 were 67, 10, 33, 10, 38, 10, 32, and 42 respectively. The combination of recursive feature elimination and random forest classifier achieved the highest average cross-validation score (0.87), Stepwise forward selection with k nearest neighbors, and L2 with support vector machine yielded the highest accuracy (0.77) in the validation set. Stepwise forward selection with k nearest neighbors also had the highest AUC with 0.72[95% CI, 0.77-0.61] while select percentile with stochastic gradient descent performed the worst with an AUC of 0.53[95% CI, 0.56-0.31] in the validation set.

Conclusions: The preliminary study results showed that the combination of feature selection methods and classifier in ML algorithm significantly impacted the prediction performance of PET radiomic-based ML models. Therefore, different feature selection methods, as well as classifiers inclusion, could prove beneficial for determining the optimal radiomic-based prediction model and eventually for the successful generation of PET radiomic biomarkers in lung cancer mutations.

• Download figure
• Open in new tab
• Download powerpoint