⁶⁸Ga-DOTATATE PET/CT Interobserver Agreement for Neuroendocrine Tumor Assessment: Results of a Prospective Study on 50 Patients

Patients and Image Acquisition

From June 2013 until March 2014, 50 patients with known or suspected NET of the small bowel (n = 19), pancreas (n = 14), lung (n = 4), or other location (n = 13) were prospectively recruited under a Food and Drug Administration–approved Investigational New Drug application. The patients had been referred for NET staging (n = 10) or restaging (n = 40). All had been included in a previously published study on the impact of ⁶⁸Ga-DOTATATE PET/CT on patient management (8). The prospective study was approved by the University of California, Los Angeles, Institutional Review Board, and all subjects gave written informed consent. The patients were prepared and the images acquired as previously described (8). In brief, ⁶⁸Ga-DOTATATE was injected intravenously at a dose of 190 ± 17 MBq (5.1 ± 0.5 mCi; range, 130–211 MBq [3.5–5.7 mCi]). A tracer uptake period of 60 min (mean, 62 ± 7 min) was allowed before imaging, which was performed using a Biograph True Point 64 or Biograph mCT device (Siemens). For anatomic correlation and identification of organ lesions, intravenous contrast medium (90–115 mL of Omnipaque 350 [GE Healthcare] at a flow rate of 2 mL/s) was administered to 47 of the 50 patients (94%). Oral contrast medium (∼600 mL of barium sulfate [Readi-Cat 2; Bracco]) was given to all patients within 1 h before the scan.

Observers

Anonymized PET/CT images (one per patient) were electronically submitted to 7 nuclear medicine physicians from 5 centers in Europe (n = 4) or North America (n = 1). The data included standard DICOM files of CT, attenuation-corrected PET, and uncorrected PET images. All physicians had at least 5 y of experience with interpreting oncologic ¹⁸F-FDG PET/CT images. All centers had been performing ⁶⁸Ga-DOTATATE PET/CT on a regular basis for at least 2 y, and all physicians had experience with ⁶⁸Ga-DOTATATE PET/CT. The observers reported their number of years of experience and the number of ⁶⁸Ga-DOTATATE PET/CT scans that they had interpreted. On the basis of these reports, the observers were grouped into two experience categories: low level (<500 scans or <5 y of experience with ⁶⁸Ga-DOTATATE PET/CT; n = 4) and high level (≥500 scans or ≥5 y of experience with ⁶⁸Ga-DOTATATE PET/CT; n = 3).

Visual Interpretation

Each observer received a written guide to image interpretation and two patient examples, which are shown in the supplemental materials (available at http://jnm.snmjournals.org), and reported the results in a table template. The following patient information was disclosed to each observer before image interpretation: indication (staging or restaging), sex (male or female), age (y), height (cm), weight (kg), injected dose (mCi), uptake time (min), CT protocol (contrast-enhanced or nonenhanced), prior therapy (yes or no), prior chemotherapy (yes or no), prior PRRT (yes or no), prior surgery (yes or no), and prior octreotide therapy (yes or no). The observers were masked to all other clinical data.

The images were visually interpreted for the following: overall scan result for presence or absence of disease, SSR density in NET tissue (none, 0; low, 1; intermediate, 2; or high, 3), indication for PRRT (yes or no), organs affected (yes or no), number of organs affected (0, 1, 2, 3, 4, or ≥5), number of organ metastases detected (0, 1, 2, 3, 4, or ≥5), lymph nodes affected (yes or no), number of lymph node regions affected (0, 1, 2, 3, 4, or ≥5), and number of lymph node metastases detected (0, 1, 2, 3, 4, or ≥5). The criteria for a PRRT indication were, among others, intense tracer uptake by tumor lesions and metastatic spread.

SUV Measurement

Each observer recorded series number, image number, location, and SUV_max for up to 3 target lesions. The lesions were chosen from each diseased organ system following predefined criteria as outlined in the written guide to image interpretation.

Each observer measured SUV_max and SUV_mean using a 5-cm-diameter circular region of interest in a lesion-free location in the right hepatic lobe and a 2-cm-diameter circular region of interest in lesion-free splenic parenchyma. To ascertain consistency, the observers were provided with a PET/CT dataset and SUV data for one test patient for comparison.

Reference Standard and Statistical Analyses

Data acquisition and analyses were performed prospectively. Image interpretations by a University of California, Los Angeles, physician who had access to all baseline and follow-up clinical information served as the reference standard. For binary data, agreement between each observer and the reference observer was evaluated using the Cohen κ statistic (9). Overall agreement using pooled observer data was evaluated using generalized estimation equations (10). For nonbinary data, agreement among observers was evaluated by the intraclass correlation coefficient (ICC) using a 2-way mixed model for absolute agreement (single measures) (11). To calculate ICC for tumor SUV_max, one target lesion, that is, the lesion reported by most observers, was chosen per patient. Ninety-five percent confidence intervals (95%CIs) are reported for κ and ICC values. Interpretation of κ and ICC was based on a classification provided by Landis and Koch (12): 0.0, poor; 0.0–0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1.00, almost perfect reproducibility.

Discrepancies in quantitative ratings among observers were expressed as mean difference (Δ) ± SD. Statistical analyses were performed using R software (R Core Team 2015; R Foundation for Statistical Computing) with the package “irr” (version 0.84) for generalized estimation equation modeling and SPSS (version 15.0; SPSS Inc.) for all other statistical analyses.