Abstract
The goals of this study were to examine the real-world effectiveness of PET in avoiding unnecessary surgery for newly diagnosed patients with non–small cell lung cancer. Methods: A cohort of 2,977 veterans with non–small cell lung cancer between 1997 and 2009 were assessed for use of PET during staging and treatment planning. The subgroup of 976 patients who underwent resection was assessed for several outcomes, including pathologic evidence of mediastinal lymph node involvement, distant metastasis, and 12-mo mortality. We anticipated that PET may have been performed selectively on the basis of unobserved characteristics (e.g., providers ordered PET when they suspected disseminated disease). Therefore, we conducted an instrumental variable analysis, in addition to conventional multivariate logistic regression, to reduce the influence of this potential bias. This type of analysis attempts to identify an additional variable that is related to receipt of treatment but not causally associated with the outcome of interest, similar to randomized assignment. The instrument here was calendar time. This analysis can be informative when patients do not receive the treatment that the instrument suggests they “should” have received. Results: Overall, 30.3% of patients who went to surgery were found to have evidence of metastasis uncovered during the procedure or within 12 mo, indicating that nearly one third of patients underwent surgery unnecessarily. The use of preoperative PET increased substantially over the study period, from 9% to 91%. In conventional multivariate analyses, PET use was not associated with a decrease in unnecessary surgery (odds ratio, 0.87; 95% confidence interval, 0.66–1.16; P = 0.351). However, a reduction in unnecessary surgery (odds ratio, 0.53; 95% confidence interval, 0.34–0.82; P = 0.004) was identified in the instrumental variable analyses, which attempted to account for potentially unobserved confounding. Conclusion: PET has now become routine in preoperative staging and treatment planning in the community and appears to be beneficial in avoiding unnecessary surgery. Evaluating the effectiveness of PET appears to be influenced by potentially unmeasured adverse selection of patients, especially when PET first began to be disseminated in the community.
PET has been demonstrated to more accurately assign tumor stage (T stage) and is more sensitive in detecting mediastinal lymph node involvement (N2- or N3-stage disease) in the evaluation of non–small cell lung cancer (NSCLC) (1). Accurate staging for lung cancer is critical to identify patients with advanced disease because surgical resection provides little long-term clinical benefit to the patient if the disease is not confined to the lungs.
Several small, randomized trials have reported promising findings about the efficacy of PET in identifying occult mediastinal lymph node involvement and in reducing the frequency with which patients receive potentially unnecessary surgery for advanced-stage disease. Unnecessary surgery has been defined as resection in patients with N2 or N3 mediastinal lymph node involvement, patients with evidence of extranodal metastases, or patients succumbing to death within 12 mo of surgery (2). Fischer et al. observed that 13 (25%) of 52 patients who went to surgery after undergoing PET were found to have received unnecessary surgery, compared with 38 (53%) of 71 patients who received conventional staging (2,3). This study suggested that to avoid a single unnecessary surgery, 4.9 patients (i.e., the number needed to treat) had to undergo PET as part of preoperative staging, and the reduction in unnecessary surgery associated with preoperative PET staging may save costs (4). Similarly, in a prior randomized trial, van Tinteren et al. found that 19 (32%) of 60 patients who went to surgery after undergoing PET received unnecessary surgery, compared with 39 (50%) of 78 patients who received conventional staging (5). Both trials also reported a reduction in the number of patients initially undergoing surgery because of abnormalities identified by PET. This study also concluded that preoperative staging with PET saved costs (6). However, a study by Viney et al. reported that PET did not lead to a reduction in the proportion of patients referred to surgery (7).
The effectiveness of PET in community practice has not been evaluated. A health technology assessment performed by the United Kingdom’s National Health Service concluded that “There have been no clinical studies which demonstrate that FDG-PET leads to improvement in patient outcomes” and highlighted the importance of findings about the value of PET in the diagnostic pathway for typical patients (8,9). In this report, we examine the use of PET in the preoperative staging process in a large community cohort of patients.
MATERIALS AND METHODS
Patients and Data Collection
We identified veterans cared for in the Department of Veterans Affairs (VA) Northwest Health Network of the Veterans Health Administration who were diagnosed with NSCLC between January 1, 1997, and December 31, 2009. The VA Northwest Health Network comprises 8 medical centers and 30 community-based outpatient clinics across a multistate area that includes Washington, Alaska, Oregon, Idaho, a county in Western Montana, and a county in Northern California. Newly diagnosed patients were identified through the VA’s cancer registry program (10,11). The VA Northwest Health Network cancer registry has been previously compared with the Surveillance, Epidemiology, and End Results Puget Sound cancer registry, with the VA Northwest Health Network registry having more complete staging and treatment information (12). All study procedures, including a waiver of consent to extract information from the VA’s registry and data warehouse, were approved by the institutional review boards of the VA Puget Sound Healthcare System and the Fred Hutchinson Cancer Research Center.
Patients older than age 100 y were excluded, as were cases identified as diagnosed only on the death certificate or at the time of autopsy. The histologic types of NSCLC included adenocarcinoma, squamous cell carcinoma, other NSCLC (including bronchioloalveolar carcinoma, large cell carcinoma, mixed types, and other types), and NSCLC not otherwise specified (13). Detailed staging information is abstracted by tumor registrars based at each of the medical centers in the VA network. Staging information follows the Committee on Cancer’s Facility Oncology Registry Data Standards.
The VA’s electronic medical record was queried to identify staging procedures, including PET imaging and mediastinoscopy, as well as type of treatment, including pneumonectomy, lobectomy, wedge resection or other thoracic surgical procedures, radiotherapy, and chemotherapy. PET imaging tests and mediastinoscopy occurring between 180 d before and 180 d after the date of diagnosis were assumed to be part of the staging process as long as the procedure was before the date of surgery for those patients who underwent resection. Nearly all PET procedures included concurrent codes for receipt of CT; however, because we were unable to distinguish whether a facility had integrated multidetector PET/CT scanners, we used exposure to PET in any form as our primary outcome. We note that procedures performed by outside facilities but paid for by the VA are included in the VA’s electronic medical record as fee-basis procedures. During this study, all PET scans were performed by community facilities, as none of the VA facilities included in this analysis owned dedicated PET equipment during this period. Preexisting comorbidity at the time of diagnosis was categorized using inpatient and outpatient records and the patient’s “problem list” from the VA electronic medical record based on the Charlson comorbidity index (14). All patients were followed for at least 12 mo from the date of diagnosis to identify any deaths occurring in that period.
Data Analysis
Trends in patient characteristics and patterns of PET use were examined over the study period for all newly diagnosed patients. Descriptive statistics, including χ2 tests, were compared to identify differences in patient characteristics between those who underwent PET and those who did not, as well as to identify whether patient and treatment characteristics changed over the study period. Multivariate logistic regression adjusting for age, marital status, comorbidity, and clinical stage was used to explore the likelihood of resection over time.
Our primary outcome variable among the subgroup of patients who underwent resection was unnecessary surgery based on the 12-mo definition established by Fischer et al. (3). Patients were defined as receiving unnecessary surgery if pathology records identified evidence of extranodal lymph involvement or metastasis or if the patient died within 12 mo. Although logistic regression models are common in the clinical literature, odds ratios overestimate the actual difference in risks when the outcome of interest is common (15,16). Because our outcome of interest—unnecessary surgery—was common, we report both the more familiar odds ratios and the adjusted risk differences using a risk regression approach (17). Covariates included patient age, marital status, race, calendar year, and count of severe comorbid conditions. Indicators for chronic obstructive pulmonary disease, congestive heart failure, and an indicator for either renal failure or liver disease were also included in the model. Confidence intervals for the adjusted risk differences were obtained using bootstrapping with 5,000 replicates.
Our primary analysis included an indicator for whether a mediastinoscopy procedure was performed. In sensitivity analyses, we removed this variable to explore whether receipt of mediastinoscopy confounded the association between PET and unnecessary surgery.
An instrumental variable analysis was also planned a priori to attempt to adjust for residual confounding associated with potential selection of patients to receive PET in this observational setting. Calendar year was a strong predictor of PET and served as our instrument, acting as a variable determining assignment to PET, similar to randomized treatment assignment in a trial (i.e., more patients in early years were systematically assigned to not receive PET). A valid instrument should not be correlated with the outcome of interest except through the mechanism of exposure (i.e., increasing use of PET). We explored the validity of this independence assumption by examining characteristics of patients over time and by assessing whether temporal changes in other lung cancer outcomes were evident in a related population that was not included in our main analysis. This analysis focused on assessing trends in outcomes over time among patients with stage IV disease, as any temporal improvements in this population would be an indicator that outcomes were changing more generally over time, providing evidence for or against the appropriateness of this instrument. Detailed data are presented in a supplemental appendix (available at http://jnm.snmjournals.org) based on recommendations by Brookhart et al. (18).
We used a 2-stage residual inclusion approach in which a first-stage model was performed that predicted the probability of undergoing PET as a function of calendar year and other patient covariates. The residuals from this model were then included in the second-stage model as an additional covariate alongside the primary independent variable—receipt of a preoperative PET scan (19,20). All variables from the conventional model were included in the second-stage model, with the exception of the instrument (i.e., calendar year), which was included in only the first-stage model. All analyses were conducted in STATA, version 11.0. (STATA Corp.).
RESULTS
We identified 2,977 veterans with newly diagnosed NSCLC during the study period. Most of patients were male (97.8%), white (89.3%), and either current smokers (58.3%) or former smokers (36.1%). The average age at diagnosis was 68.2 y, with some differences observed in age at diagnosis across the study period (Table 1). In later years of the study there were more patients over age 75 y and more patients under age 65 y than in the early study period (P < 0.001). Comorbidity was common, with 71.1% of subjects having at least one coexisting severe chronic condition. In the earlier years of the study, 1997–2000, 6% (52/821) of newly diagnosed patients underwent PET, compared with 61% (280/457) in the later study years.
Characteristics of the 2,977 Veterans Newly Diagnosed with Lung Cancer
Among all newly diagnosed patients, 976 (32.8%) went on to surgery (Table 1). The frequency of surgical resection declined slightly over time. Among patients diagnosed in early years of the study, when PET was rare, 34.4% (605/1,758) underwent resection, compared with 30.4% (371/1,219) of subjects diagnosed in later years (P = 0.023). After adjustment for age, marital status, comorbidity, and clinical stage, the decline in the frequency of surgery was −6.3% (95% confidence interval [CI], −12.0 to −0.6; P = 0.003) for early years compared with later years, when PET was more common.
Among the subgroup of 976 patients who went to surgery, 56.3% underwent PET preoperatively. The mean number of days the PET scan occurred before surgery was 42, with a median of 35. Calendar year was the only predictor we identified associated with PET use (Table 2). A noticeable increase in preoperative PET staging among surgery patients was observed: 9.2% (1997–2000) vs. 91.2% (2008–2009). The P value for a trend test across all years was less than 0.001. Notably, there were no significant differences in PET use by patient age (P = 0.082), race (P = 0.389), or comorbidity (P = 0.274).
Characteristics of the 976 Patients Undergoing Resection by Preoperative PET Use
Overall, 54.1% (1,611/2,977) of patients died within 12 mo of diagnosis (Table 3). Among the subgroup of patients who went to resection, 20.6% (201/976) had died within 12 mo of diagnosis. In unadjusted analyses, there was no difference in 12-mo mortality among surgery patients who underwent preoperative PET and those who did not (P = 0.419). Subjects who underwent preoperative PET were less likely to have evidence of metastasis (P = 0.018); however, the frequency of advanced-stage disease was not statistically significant (P = 0.062) among those who received preoperative PET compared with those who did not receive PET.
Surgical Outcomes of Patients Undergoing Resection by Preoperative PET Use
Overall, 30.3% (296/976) of patients underwent unnecessary surgery (Table 3). In univariate analyses, there was no difference in the frequency of unnecessary surgery between patients who received preoperative PET (29.9% [164/549]) and those who did not (30.9% [132/427]; P = 0.726). Receipt of a preoperative PET scan was not associated with a difference in the likelihood of unnecessary surgery (Table 4) in the conventional regression analysis adjusting for patient age, comorbidity, marital status, and mediastinoscopy (absolute risk difference of −3.0%; 95% CI, −9.1 to 3.1; P = 0.335). Findings from sensitivity analyses excluding mediastinoscopy were similar.
Effect of Receiving Preoperative PET on Chances of Unnecessary Surgery
In the first-stage model of the instrumental variable analysis, calendar time was strongly predictive of receipt of PET (Chow F statistic = 442.8, P < 0.001) but was not strongly correlated with other outcomes. For example, the Chow F statistic for the association between time and survival among patients with stage IV disease was 1.28 (P = 0.258). The hazard ratios for overall survival for 2005–2007 and 2008–2009 relative to 1997–2000 were 1.00 (P = 0.965) and 1.01 (P = 0.796), respectively, suggesting that outcomes were not changing more generally over the study period. The supplemental appendix has additional details about the validity of the instrumental variable assumptions.
When the residuals from the instrument were included in the analysis (Table 4), receipt of preoperative PET was found to be protective against the likelihood of unnecessary surgery. The odds ratio was 0.53 (95% CI, 0.34–0.82; P = 0.004), which corresponds to an adjusted risk difference of −10.2 absolute percentage points compared with if PET had not been used (95% CI, −20.2 to −0.03; P = 0.044).
DISCUSSION
In this community-based evaluation of veterans with newly diagnosed NSCLC, the use of preoperative PET among patients going on to surgery increased substantially over the past decade, and now PET scans are nearly universally obtained before surgical resection. Quantifying the benefit of this increase in PET use in the real world is challenging because many factors may influence outcomes, including the changing quality of PET imaging over time, variability by radiology and surgical providers, and potential selection issues in who received PET, especially when availability of PET was rare in the late 1990s. For example, studies of the diffusion of PET in cancer staging have reported considerable variation in PET use, with higher rates of PET among whites and higher socioeconomic groups (21). Notably, in our analysis we did not observe differences in PET use by race, patient age, or other observable demographic factors (Table 2).
We note that in conventional analyses, which adjusted for a limited number of observed clinical and demographic characteristics, the receipt of a preoperative PET scan was not strongly associated with avoidance of unnecessary surgery. However, the temporal variation in PET use provides a natural experiment to evaluate and reduce potential selection factors for PET use, such as the likelihood that a provider insisted that a patient with an uncertain or suggestive CT finding also receive a PET scan.
Although the odds ratio is familiar, this statistic cannot be directly interpreted as the relative risk or a ratio of probabilities when likelihood of the outcome is common. In the adjusted-risk-differences approach, our finding that preoperative PET use was associated with an absolute reduction of −10.2% is consistent with prior randomized trials (2,3,5). Using this approach, we estimate that 38.2% of patients undergoing resection would have received surgery unnecessarily if PET had not been available to anyone during the study period (17). This estimate highlights that the observed rate of 30.3% (296/976) reflects a substantial improvement in patient outcomes.
This community-based analysis differs from prior randomized trials in several ways. First, all veterans diagnosed with lung cancer who underwent resection were included. This is a key strength of our study, as our findings reflect a large cohort of typical patients in the community with a variety of comorbidities and not a selected subset of recruited patients who consented to be in a trial in which all care was carefully scrutinized according to study protocols. For example, the Northwest VA Network did not own any PET imaging equipment during the study period; thus, all PET scans were contracted by a variety of community providers and the quality of imaging tests may have varied.
Second, we used an instrumental variable approach to address unobserved confounding. This analytic method is analogous to systematic treatment assignment in a randomized trial. Here, we used the instrument calendar time, which systematically “assigned” patients to more frequently receive PET in later years of the study. Compliance with the assigned treatment is often imperfect, even in a trial setting. This gap between treatment assigned and treatment received provides valuable information. In a trial, this information is known to be associated with bias; thus, randomized studies use intent-to-treat analyses rather analyzing data as treated. In the observational setting, deviations from a subject’s expected or instrumental-assigned treatment provide a way to control for potential selection bias. By including these residuals in the analysis, we incorporate a measure of unobserved characteristics that may have influenced selection of PET. Although inclusion of instrumental variable residuals has been demonstrated in simulations to consistently remove potential confounding due to selection when it was present (19,20), this is no guarantee that all confounding between the outcome and PET use has been accounted for. Although we did not observe improvements in the proportion of patients diagnosed with stage IV disease or changes in survival over time (supplemental appendix), it is possible that unobserved contemporaneous improvements in care, leading to reduced rates of unnecessary surgery, could be independent of the probability of referral to PET and may explain a portion of the improvement in outcomes we have attributed to the use of PET.
Our study highlights the challenge of quantifying the real-world effectiveness of nuclear medicine procedures on subsequent patient outcomes, noting the importance of considering selection biases among patients who do and do not undergo advanced imaging.
The retrospective reliance on available data in the VA’s electronic medical record is a limitation of the study. These data were not intended to quantify the relationship between PET and unnecessary surgery. Our broad window capturing use of PET (180 d before diagnosis, up to the time of surgery or 180 d after diagnosis if the patient did not undergo resection) was selected to ensure that we captured all PET use. However, we may have attributed to staging a small number of scans that were ordered for other reasons. We used electronic data to define unnecessary surgery, which included stage IIIA N2 disease from cancer registry records. However, some patients with suspected N2 disease may still be appropriate candidates for resection. The ideal study would quantify the number of patients who were considered for surgery but did not undergo it because of a suggestive PET finding, such as was examined in the randomized trials. Such an analysis is not possible in electronic records as it is not feasible to identify these patients. Thus, our primary analysis was performed only among the subgroup of patients who could be identified as undergoing resection. This design, which focuses on outcomes among surgical patients only, allows us to quantify at the population level how many patients avoided resection because of PET; however, we are not able to specifically identify which patients avoided an unnecessary resection among the entire population of unresected lung cancer patients. Also, because the data did not include detailed text fields or notes, we were not able to confirm the findings of the PET scans or review surgical reports to determine whether PET influenced the surgical decision. An additional limitation is that any service a VA patient receives that is covered by non-VA insurance (such as Medicare or a veteran’s private insurance) may not be captured by the VA’s electronic medical record. In a review of lung cancer patients over age 65 y linked to Medicare data, Keating et al. confirmed that VA records for chemotherapy and radiotherapy were 98% and 99% sensitive to capturing receipt of those therapies (22). This suggests that veterans diagnosed with lung cancer at a VA facility appear to remain in the VA for nearly all of their cancer care.
CONCLUSION
PET use has increased substantially over the past decade in this community setting. In current practice, nearly all patients who undergo lung resection for newly diagnosed lung cancer (91.2%) undergo preoperative PET. Conventional multivariate analysis in this observational setting did not identify a direct association between PET use and a reduction in the proportion of patients found to have occult metastases during or shortly after surgery. The use of instrumental variable analysis did identify a reduction in unnecessary surgery associated with receipt of PET. This suggests that selection bias may have been common when PET first began to be disseminated, with PET initially potentially offered preferentially to patients suspected of having distant occult metastasis. More recently, PET appears to be used routinely in staging for multiple reasons including clarifying regional lymph node status, not primarily for determining distant metastases. Evaluating the effectiveness of advanced imaging in community practice is challenging and requires careful attention to potential selection bias.
DISCLOSURE
The costs of publication of this article were defrayed in part by the payment of page charges. Therefore, and solely to indicate this fact, this article is hereby marked “advertisement” in accordance with 18 USC section 1734. This material is the result of work supported by a grant from the National Cancer Institute for the “ADVancing Innovative Comparative Effectiveness research in cancer diagnostics (ADVICE)” study, 1RC2CA148433-01; by resources from the VA Puget Sound Health Care System, Seattle, Washington; and in part by a Health Services Research and Development Career Development Award from the Department of Veterans Affairs and resources from the Portland VA Medical Center, Portland, Oregon. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the U.S. Government. No other potential conflict of interest relevant to this article was reported.
Footnotes
Published online Jan. 21, 2014.
- © 2014 by the Society of Nuclear Medicine and Molecular Imaging, Inc.
REFERENCES
- Received for publication April 1, 2013.
- Accepted for publication September 30, 2013.
Jump to section
Related Articles
Cited By...
- No citing articles found.