Test–Retest Reproducibility of ¹⁸F-MPPF PET in Healthy Humans: A Reliability Study

Tracer Synthesis.

The ¹⁸F-MPPF was obtained by nucleophilic fluoration on a nitro precursor with a radiochemical yield of 20%–25% at the end of synthesis and a specific activity of 37–111 GBq/μmol (18,19).

Scanning Procedure.

Subjects underwent 2 ¹⁸F-MPPF PET scans (test and retest) separated by a 6-mo period (mean delay ± SD, 27 ± 2 wk), randomly distributed along the year. Each PET session began at 1 pm. The PET scan acquisition, correction, and reconstruction procedures followed those described in (12). PET scans were obtained with a CTI Exact HR+ camera for 60 min after the injection of 2.7 MBq/kg (mean total dose ± SD, 169 ± 30 MBq) of ¹⁸F-MPPF. There was no significant difference (P = 0.93) in the paired t test between the injected dose of the test and retest scans.

Simulated Data.

Following the methodology for the simulation of realistic PET described in (20), we performed a joint simulation of a test–retest study. Ten individual different numeric brains associated with 10 different sets of regional time–activity curves were used to simulate 10 realizations of ¹⁸F-MPPF PET dynamic acquisition. The simulations were repeated to obtain the retest simulated data. The input time–activity curves used for the test and retest simulated acquisition set were identical for each subject. Therefore, the only difference between the simulated test and retest datasets was due to the degradation induced by the physical acquisition processes. Thus, physical variability can be compared with the measured one estimated from the actual data that include all sources of variability.

Modeling.

The binding parameters of the tracer were estimated according to the 3-compartiment simplified reference tissue model (SRTM) (21) and with the Logan graphical method (22). The SRTM method lies on an analytic solution of the compartment model (Eq. 1) and allows estimating 3 indices—R₁, k₂, and BP—without requirement of an arterial sampling input function. The SRTM works under several assumptions: (a) the existence of a reference tissue region with negligible concentration of specific binding sites, (b) the magnitude of nonspecific binding is the same in the reference and in the target regions, (c) the distribution volumes (DVs) in the free and nonspecific compartments are the same in the reference and in the target regions, and (d) the exchanges between the free and unspecific binding compartments are rapid.

The analytic solution of the partial derivatives system is of the following form:Eq. 1where C_ref and C_roi are the PET time–activity curves of the chosen reference region and the target region of interest (ROI), R₁ is the ratio of the plasma-to-brain transport constant in the target region and in the reference region (R₁ = k_1roi/k_1ref), k₂ is the tracer's efflux in the vascular system, and BP is the binding potential of the tracer, defined as the ratio of available receptor density to the receptor affinity (BP = B_max/K_d). For the ¹⁸F-MPPF, the cerebellum was taken as the reference region as it is considered devoid of binding sites (11). The Logan model is based on the parametric plot involving the PET time–activity curve of the reference region and the target region. As for the SRTM, the cerebellum was taken as the reference region. The Logan model provides the DV of the target ROI, given by the slope of the linear regression after equilibrium.

Two approaches for binding index reliability analysis were used: The first is based on mean ROI activity curve measurements (ROI Analysis) and the second is based on a voxelwise computation leading to statistical parametric maps (Parametric Image Analysis).

For reorientation and registration purposes, mean ¹⁸F-MPPF static images were created by summing the individual dynamic frames from 0 to 60 min after injection. The MR image was coregistered with the static ¹⁸F-MPPF image by an automated method using Mutual Information criteria (Statistical Parametric Mapping [SPM], Welcome Department of Cognitive Neurology, London, U.K.). On the coregistered MRI, a large ROI was outlined on the cerebellum and used as a unique reference region for the simplified models.

ROI Analysis.

The anatomic target ROIs were drawn manually on the coregistered MRI. Four hundred ROIs were drawn and regrouped into anatomic volumes of interest (VOIs) to describe a group of regions from the limbic system, known to be rich in 5-HT_1A receptors—that is, left and right hippocampi, amygdala, enthorinal cortex, parahippocampal gyri, anterior and posterior cingulate gyri, insula, temporal poles, and temporal cortex—and a second group of other cortical regions: left and right temporal neocortex, lateral occipitotemporal gyrus, frontal gyrus, prefrontal cortex, inferior and superior parietal cortices, occipital cortex, pole and gyrus, cerebellum. As raphe nuclei cannot be identified on MRI, this region was outlined directly on the static ¹⁸F-MPPF image by thresholding the activity at 80% of the local maximum in the brain stem. This region was visualized a posteriori on the MRI to check for its proper location in the periacqueductal gray matter. Time–activity curves were measured from the dynamic PET using the set of ROIs. The measured time–activity curves were used to derive regional values of R₁, k₂, and BP and Logan DV for each ROI.

Parametric Image Analysis.

From individual voxel time–activity curves, a parametric image of R₁, k₂, and BP was computed for the SRTM, and a parametric image of DV was computed for the Logan model. Individual parametric images were then transformed into a standard space using the nonlinear transformation matrix derived from the spatial normalization of the individual's MR image to the T1 MRI default template (Montreal Neurological Institute template of the International Consortium for Brain Mapping Project) with SPM. The visual inspection of the spatially registered images, particularly for subcortical structures, confirmed the accuracy of the spatial normalization. Normalized parametric images were smoothed using an 8 × 8 × 8 mm³ full width at half maximum isotropic gaussian kernel to account for the interindividual anatomy variability and to improve the sensitivity of the statistical analysis.

Reproducibility Indices.

Reliability of the R₁, k₂, and BP indices issued from the SRTM, and the DV index issued from Logan model, were assessed by computation of 3 characteristic parameters from the test–retest measurements:

• The percentage change in mean (bias): the percentage change calculated as the difference between test and retest values divided by the test value. This index includes random changes and systematic biologic error;

• The typical error or within-subject SD of the bias: We expressed the typical error as the percentage of the mean;

• The intraclass correlation coefficient (ICC) estimates the respect of the rank in a test–retest study: It depends on the size and the quality of the sample in the population. ICC = (MSBS − MSWS)/(MSBS + MSWS), where MSBS is the mean sum of squares between subjects, and MSWS is the mean sum of squares within subjects.

Because these variables have residuals that may be proportional to their respective mean, their computation is performed from the logarithmic transform of the variables as suggested in (23).

Statistical Inference.

For the ROI analysis, mean regional binding index values were considered as independent measures.

For the voxel-based analysis, SPM99 was used on the normalized smoothed parametric images of the 10 subjects. Statistical parametric maps of the t statistic (SPM{t}) were computed with a threshold of P = 0.001 uncorrected at the voxel level. Significant clusters were selected at a corrected cluster level of P < 0.05 determined from a joint probability of peak height and cluster size (“Family-Wise Error” (24)).

Actual BP.

As shown in Table 2 (top) test and retest, mean regional BP values range from 0.28 ± 0.08 in the raphe to 1.47 ± 0.16 in the hippocampus. The percentage changes between test and retest values ranges from −1.15% (anterior cingulum) to 4.80% (enthorinal cortex), with a mean typical error of 7.75% in the limbic area and 7.71% in other regions. The maximal typical error is in the raphe nucleus (14.97%) and the minimal error is in the parahippocampal gryus (4.67%). The ICC goes from 0.50 (anterior cingulum) to 0.93 (inferior parietal cortex). Mean ICC values are 0.69 in the limbic area and 0.84 in other cortical regions. None of the regional BP differences was found to be statistically significant with a paired t test comparison.

Actual R₁ (Table 2, Middle).

Reproducibility of the relative perfusion parameter (R₁) in the ROIs is also excellent, with an average percentage changes of 1.63% in the limbic area (from −5.03% in the temporal pole to 5.21% in the hippocampus), and a mean typical error of 5.77%. The typical error of the R₁ parameter is generally inferior to the error of the BP parameter, going from 4.60% (raphe) to 7.86% (hippocampus). The average change is −3.43% in the other cortical regions, with a mean typical error of 5.98%. The ICC revealed very puzzling values, from −0.17 in the prefrontal cortex to 0.67 in the temporal cortex. The mean ICC is 0.37 in the limbic areas and 0.25 in the other cortical regions.

Actual k₂ (Table 2, Bottom).

The k₂ values go from 0.11 ± 0.01 min⁻¹ in the raphe, to 0.35 ± 0.05 min⁻¹ in the occipital gyrus. The percentage changes in the mean between test and retest series go from −4.42% in the enthorinal cortex to +2.25% in the prefrontal cortex. In the limbic areas, the average percentage change in mean was −0.50%, very similar to that of the other regions (−0.51%). The mean typical error is 9.50% in the limbic regions and 10.53% in the other cortical regions. The mean ICC value is 0.40, with a large range from −0.03 in the anterior cingulated cortex to 0.77 in the parahippocampal gyrus. In other regions, the mean ICC value is 0.31, with a range from 0.15 in the occipital cortex to 0.42 in the occipital gyrus.

Simulated Data.

Simulated data have an excellent ICC (>0.95), a mean percentage change in BP values around −1%, with a mean typical error of 2.5% (Table 3, top). The R₁ parameter shows a percentage change in the mean around zero and a typical error of 1.79% in the limbic areas and 0.85% in the other regions (Table 3, middle). The k₂ of simulated data has a percentage change in the mean of <1% and a typical error of <3% (Table 3, bottom).

Bias and Typical Error.

The mean percentage changes for the BP are inferior in the limbic areas (<1%) to that in the other cortical regions (>2%); however, the typical errors are similar (around 7%). The simulated data predicted a mean percentage change of around 1%, with a typical error of 2%. Thus, we can conclude in favor of a better stability of the test–retest measurement close to the ideal—with a biologic uncertainty equivalent in regions rich and poor in 5-HT_1A receptors—larger than the simple measurement error due to the PET image formation process. The R₁ parameter has a variability around ±5% in rich regions, which systematically increases (values from 2% to 5%) in poor regions. For the 2 classes of regions, the typical error is close to 6%. The simulation study indicates that the reproducibility should be near zero and the typical error should be between 1% and 2%. Because R₁ is related to cerebral blood flow, we can state the hypothesis that the regional cerebral blood flow was greater during the retest scan than during the first scan. Because this finding is not observable in the BP results, we can state that modeled parameters are effectively identified independently. Finally, the k₂ parameter has a very good stability (1% in the mean) for a typical error of measurement close to 10%, whereas simulation predicted a 3% typical error in the absence of intraindividual variability. In conclusion with the SRTM, the R₁ is the estimated parameter that presents the lowest measurement error (5%), followed by the BP (7%) and by the k₂ (10%). The Logan model shows a reliability result similar to the BP reliability. Precisely, DV has a higher variability but a lower typical error than the BP.

Variability.

The ICC is a measure of the correlation between the values obtained with 2 methods within the same subject (25). It is used as an index of reliability of the test–retest measurements and combines information of the systemic difference between methods (test and retest) and of the measurement variations. In a PET study on ¹¹C-WAY 100635, ICC values above 0.50 and 0.75 were considered as acceptable and excellent, respectively (16). In our study, the ICC values of BP and DV were poor (<0.5) to excellent (>0.75). On average, the ICC is slightly inferior in the limbic area than in the other cortical regions. Because this parameter is representative of the individual stability between test and retest scans, it appeared that it was more variable in regions with high 5-HT_1A receptor densities than in poor regions. The SDs of the value in test–retest are similar in the limbic area and in the other regions (0.12 for the BP), so the difference in the ICC is due to a higher intrasubject variability in rich regions than in poor regions.

This phenomenon must be considered when individual test–retest results are examined, but it does not affect the reproducibility of a group comparison according to Parsey et al. (2000) and Hirvonen et al. (2006) with ¹¹C-WAY 100635. Many studies reported moderate ICC values: ICC values are higher with the SRTM than with graphical or nonlinear fitting techniques with peripherical arterial blood function (14–16,26,27).

Potential Use for Group Comparison.

The results of the reliability study may help to determine the minimal size of the sample of a study as far as the delay between consecutive pairs of trials is similar to the delay of the reliability study (around 6 mo in that experiment). Under that condition of delay, we take into account the 2 components of the typical error: the experimental error and the biologic variability. For a test–retest study with short delay, the biologic error may be different and ideally reduced to zero. In that case, variability of measurements is only due to experimental error rather than close to the typical error found in the simulated data presented in this article. But in specific cases of long-term clinical studies, the natural biologic variability of the control population has to be known to determine the minimal sample size allowing optimal conditions for detection. In crossover studies, a simplified formula useful to fix sample size is n = 8 s²/d², where “d” is the minimal difference to be detected between pre- and post test acquisitions, and “s” is the typical error found in the reliability study (23). The coefficient 8 is an approximation of 2 times the inverse Student distribution for a confidence level of 95%. As an example, expecting a difference of 5% of the variation of BP in the hippocampus (typical error of 7.4% in our reliability study) in a crossover test–retest study will require a sample size of 17 subjects for a statistical power of 95%. That sample size must be multiplied by 4 when another independent group is used as control. When the expected differences are much larger than noise—for example, looking for a difference of 15% in BP in the hippocampus—only 2 subjects are required (8 × 7.4²/15² = 1.9). In that case, the only restriction is to ensure that the selected 2 subjects are representative of a wider population. The major advantage of conducting a reliability study is that it allows performing a crossover study without a healthy control group, just from the knowledge of the typical error of the present reliability study.

Assessing for Individual Measurement.

In that case, the typical error is used quite differently. One approach consists of assessing the difference between 2 scans: if the difference exceeds a confidence interval based on the typical error found in the reliability study (mean ± 2 SD for a 95% likelihood). Another approach, more powerful and less restrictive, consists of establishing whether an expected difference between test and retest acquisitions is exceeded by measurement of an individual subject, with reasonable likelihood. That approach forces one to make an estimation of what smallest clinically important changes between measurements would have a significant importance. So, a priori knowledge of biologic variability is required. Let's suppose that a 10% modification of ¹⁸F-MPPF BP is an expected value for evidence of clinical variation in the serotoninergic system. With that case, if an individual patient presented a scan difference of 14% in the hippocampus, knowledge of the test–retest typical error allows evaluation of the confidence interval of the true value of changes for a determinate likelihood—that is, for an 80% likelihood, the factor to be applied to the typical error around the measurement is 1.81. With that value, the confidence interval around the patient variation of 14% will be [12.2;15.9]. We can assume that, with an 80% likelihood, true change of the measured change of 14% is greater than a 10% change, so that a change actually occurred between test and retest conditions. This approach is less conservative but is more effective and clinically practical for deciding on an effect in therapeutics.

More generally, a general usage of parametric imaging is the established individual or group comparison at a voxel level, thanks to the usual and friendly approach of SPM. In that study, we verified that the parametric images did not present significant bias between test and retest measurements: This database is then suitable for statistical inference in group comparisons and individual assessment via the general linear model.