Reduction of observer variation using matched CT-PET for lung cancer delineation: A three-dimensional analysis

Purpose: Target delineation using only CT information introduces large geometric uncertainties in radiotherapy for lung cancer. Therefore, a reduction of the delineation variability is needed. The impact of including a matched CT scan with 2-[¹⁸F]fluoro-2-deoxy-d-glucose positron emission tomography (FDG-PET) and adaptation of the delineation protocol and software on target delineation in lung cancer was evaluated in an extensive multi-institutional setting and compared with the delineations using CT only.

Methods and Materials: The study was separated into two phases. For the first phase, 11 radiation oncologists (observers) delineated the gross tumor volume (GTV), including the pathologic lymph nodes of 22 lung cancer patients (Stages I–IIIB) on CT only. For the second phase (1 year later), the same radiation oncologists delineated the GTV of the same 22 patients on a matched CT–FDG-PET scan using an adapted delineation protocol and software (according to the results of the first phase). All delineated volumes were analyzed in detail. The observer variation was computed in three dimensions by measuring the distance between the median GTV surface and each individual GTV. The variation in distance of all radiation oncologists was expressed as a standard deviation. The observer variation was evaluated for anatomic regions (lung, mediastinum, chest wall, atelectasis, and lymph nodes) and interpretation regions (agreement and disagreement; i.e., >80% vs. <80% of the radiation oncologists delineated the same structure, respectively). All radiation oncologist–computer interactions were recorded and analyzed with a tool called “Big Brother.”

Results: The overall three-dimensional observer variation was reduced from 1.0 cm (SD) for the first phase (CT only) to 0.4 cm (SD) for the second phase (matched CT–FDG-PET). The largest reduction in the observer variation was seen in the atelectasis region (SD 1.9 cm reduced to 0.5 cm). The mean ratio between the common and encompassing volume was 0.17 and 0.29 for the first and second phases, respectively. For the first phase, the common volume was 0 in 4 patients (i.e., no common point for all GTVs). In the second phase, the common volume was always >0. For all anatomic regions, the interpretation differences among the radiation oncologists were reduced. The amount of disagreement was 45% and 18% for the first and second phase, respectively. Furthermore, the mean delineation time (12 vs. 16 min, p < 0.001) and mean number of corrections (25 vs. 39, p < 0.001) were reduced in the second phase compared with the first phase.

Conclusion: For high-precision radiotherapy, the delineation of lung target volumes using only CT introduces too great a variability among radiation oncologists. Implementing matched CT–FDG-PET and adapted delineation protocol and software reduced observer variation in lung cancer delineation significantly with respect to CT only. However, the remaining observer variation was still large compared with other geometric uncertainties (setup variation and organ motion).

Introduction

The prognosis of patients with inoperable non–small-cell lung cancer treated with radiotherapy (RT) (with or without chemotherapy) is poor. The 5-year survival of these patients ranges from 0% to about 30%, dependent mainly on tumor stage and performance status (1, 2, 3, 4). With greater radiation doses, larger tumor control probability can be achieved (5, 6, 7, 8). However, the radiation dose delivered to a lung tumor is limited by the toxicity to the surrounding normal tissues. Therefore, new treatment techniques are being developed, such as intensity-modulated RT, stereotactic RT, and image-guided RT, to allow additional dose escalation compared with conventional three-dimensional (3D) conformal RT. To allow the safe delivery of very high doses to a tumor, a high level of geometric accuracy is needed. Therefore, the geometric uncertainties, such as patient setup variation, organ motion, and delineation variation, must be kept to a minimum. Much attention has already been paid to reducing the patient setup variation using setup protocols, immobilization, and portal imaging techniques (9, 10, 11, 12). The current setup variation is about 0.1–0.3 cm (standard deviation [SD]) (9, 10, 11, 12). Lung tumor motion due to breathing and heartbeat has been the topic of many current investigations (13, 14). The largest tumor motion is seen in the craniocaudal direction, for which the average amplitude for tumors in the lower and upper lobe is 1.2 cm (SD, 0.6 cm) and 0.2 cm (SD, 0.2 cm), respectively (15). Observer variation in delineation using only CT information is a major source of geometric uncertainty and has been demonstrated in several publications (16, 17, 18, 19). In particular, delineation of pathologic lymph nodes and atelectasis seems to be difficult. In these reports (16, 17, 18, 19), the results were based on a small number of observers and/or patients, and the analysis of the delineations was mostly limited to a volume comparison. No 3D information is available; therefore, it is not known how to combine observer variations in the delineation of lung cancer with the other geometric uncertainties such as setup variation and organ motion.

To reduce this observer variation is a major challenge. Bowden et al. (19) found, in a small study, that applying a delineation protocol improved delineation accuracy. The average variation of the measured gross tumor volume (GTV) was reduced from 20% without the protocol to 13% with the protocol. Their improved protocol included guidelines concerning level and window settings and tumor identification by a diagnostic radiologist. It has been demonstrated in several studies (20, 21, 22, 23, 24, 25, 26, 27, 28, 29) that implementation of 2-[¹⁸F]fluoro-2-deoxy-d-glucose positron emission tomography (FDG-PET) has a large impact on the delineated tumor volume and localization. Restaging patients’ disease after adding FDG-PET information greatly affects the treated volume. FDG-PET is currently the most sensitive noninvasive examination to identify pathologic lymph nodes (30). Furthermore, it has been claimed that FDG-PET can help to distinguish tumor from atelectasis (20, 31, 32, 33). To date, three studies have been published regarding the effect of FDG-PET on observer variation (28, 34, 35). Caldwell et al. (34) compared the GTVs of 30 patients delineated by three experienced radiation oncologists. The radiation oncologists first had to delineate the GTV on CT only. Next, they were allowed to adjust the CT-delineated GTV when a matched FDG-PET scan was added. The average ratio between the minimal and maximal delineated volume was reduced from 2.31 to 1.56 when FDG-PET was added. Ciernik et al. (28) demonstrated a reduction in delineation variability between two radiation oncologists in 39 patients with various solid tumors (six lung tumors). No details were given for the lung cancer patients. Recently, Fox et al. (35) reported that registration of FDG-PET with CT (9 patients) improved delineation consistency between two senior residents in lung cancer delineation compared with nonregistration (10 patients). For the last three mentioned studies (28, 34, 35), only simple volume comparisons were performed, and no 3D information was reported. Furthermore, no specific regions in which FDG-PET could reduce the observer variation were mentioned.

In 2002, we started a multi-institutional study to determine and reduce the observer variation in the tumor delineation of lung cancer. Because of the challenge concerning data exchange among different hospitals and systems, we developed our own infrastructure, using personal computers with custom-developed software, patient data distributed on CD, and submission of delineations over the Internet. The study was separated into two phases. In the first phase, the current delineation variability was evaluated in full three dimensions by asking 11 experienced radiation oncologists to delineate the GTV of 22 lung cancer patients on CT only. This phase was used to optimize the delineation process in terms of delineation protocol and delineation software. The acquired recommendations in the first phase were tested >1 year later in the second phase. In the second phase, the same radiation oncologists delineated the GTV of the same patients using matched CT–FDG-PET, an adapted delineation protocol, and adapted delineation software.

The results of the first (CT only) and second (matched CT–FDG-PET) phases are presented in this article.