TY - JOUR T1 - A screening method to assess the predictive power of radiomic features in PET JF - Journal of Nuclear Medicine JO - J Nucl Med SP - 61 LP - 61 VL - 60 IS - supplement 1 AU - Anne-Sophie DIRAND AU - Frederique Frouin AU - Irene Buvat Y1 - 2019/05/01 UR - http://jnm.snmjournals.org/content/60/supplement_1/61.abstract N2 - 61Objectives: Many studies are currently devoted to the design of radiomic models for a prediction or prognostic task. When no model with satisfactory performance is found, it is often difficult to know whether this is because the considered radiomic features do not include enough information relevant to the task or because of insufficient data. We propose a screening method to answer that question by accounting for the experimental conditions in the context of a classification task into 2 groups. Methods: Assuming a dataset of N patients for whom the task is to predict whether they belong to group 1 or group 2, the screening method consists in splitting the available dataset in a training set (TS, 3/4 of the data) and a validation set (VS, 1/4 of the data). A stratified K-Fold cross validation is used to build the model (TS) and estimate its performance (VS) characterized by the Youden index (YI=sensitivity+specificity-1). The process is iterated 50 times to get the average ± 1 standard deviation (SD) of YI. The whole procedure is then repeated using Nk = 40 x k patients (1≤k<N/40, k integer) and the change in SD as a function of Nk is plotted against k. The intersection of a linear fit of the points with the x-axis gives a number of patients Nc supposedly needed to predict the performance of the model at convergence (YIC), that is when YI no longer significantly increases when adding patients. The performance expected at convergence is predicted using a logarithmic fit of YI =f(Nk) extrapolated at Nk = Nc. This screening approach was tested using FDG PET datasets of lung cancer patients where the task was to predict the primary or metastatic status of lung tumors based on 42 radiomic features. We studied 36 experimental conditions differing by 1) the number of patients (40 to 360), 2) equal or different numbers of primary tumors and metastases, 3) prediction models: least absolute shrinkage and selection operator (LASSO), Logistic Regression (LR), Support Vector Machine (SVM) with LR and SVM used without and with dimensionality reduction (DR), where DR was Recursive Feature Elimination (RFE), Principal Component Analysis (PCA) or Linear Discriminant Analysis (LDA). Thirty-six synthetic PET datasets corresponding to the same experimental conditions were also created by randomly assigning the radiomic features values to 2 arbitrary groups (fake metastases and fake primary tumors) to study the behavior of the screening technique when no predictive model existed. Results: In real PET datasets, analyzing the 360 patients produced models with YI between 0.76 (PCA-SVM) and 0.81 (LASSO or LDA-SVM). Using our screening procedure, the number of patients Nc needed to predict the model performance YIC observed at convergence was accurately estimated (±20 patients) in 32 out of 36 experimental conditions. This number Nc varied as a function of the classifier, with LASSO requiring “only” 280 patients while 360 patients were needed for PCA-SVM to achieve the maximum YI. Models with DR always needed less patients than models without. The performance YIC at convergence was always within the predicted YI±1SD, with SD between 0.01 and 0.02 Youden units. In the synthetic PET datasets where no predictive model was expected, the screening method always concluded at a YI between -0.03 and 0.05, demonstrating the absence of predictive factors in the data. Conclusions: The proposed screening method determines the number of patients needed to get a sound estimate of the performance of a model and estimates that performance expressed as a narrow YI interval. It also identifies situations in which the radiomic features do not include information to classify the patients. ER -