|
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Special Contribution |
1 Department of Internal Medicine and the Massey Cancer Center, Virginia Commonwealth University, Richmond, Virginia; 2 Center for Statistical Sciences, Brown University, Providence, Rhode Island; 3 Department of Radiology, Duke University School of Medicine, Durham, North Carolina; 4 Karmanos Cancer Institute, Wayne State University, Detroit, Michigan; 5 American College of Radiology, Philadelphia, Pennsylvania; and 6 Mallinckrodt Institute of Radiology and the Alvin J. Siteman Cancer Center, Washington University School of Medicine, St. Louis, Missouri
Correspondence: For correspondence or reprints contact: Bruce E. Hillner, MD, Virginia Commonwealth University, 1101 E. Marshall St., Room 7013, Richmond, VA 23298-0170. E-mail: Hillner{at}vcu.edu
| ABSTRACT |
|---|
|
|
|---|
Key Words: PET cancer registry prospective studies
| INTRODUCTION |
|---|
|
|
|---|
Between 1998 and 2005, the Centers for Medicare and Medicaid Services (CMS) approved reimbursement of PET performed on Medicare beneficiaries for specific indications in 9 malignancies (2). In 2005, CMS indicated its intent to establish coverage for PET for essentially all other cancers and indications when the provider is participating in and patients are enrolled in one of certain prospective clinical studies ("coverage with evidence development" [CED]), including a prospective registry (2).
The National Oncologic PET Registry (NOPR) provides a mechanism for CMS coverage of PET for patients with currently noncovered cancers and indications in the context of a registry. The data from this registry will be used to evaluate the impact of PET on intended patient management across all of these cancer type/clinical indication categories, and within specific cancer type/clinical indication categories. CMS can use this information to make evidence-based decisions with regard to subsequent coverage of PET use for each cancer type/clinical indication category. The process leading to the development of the NOPR and an overview of the mechanics of registry operation have been reported recently (3). In this report, we describe the data collection and analysis plan of the NOPR.
| METHODS |
|---|
|
|
|---|
The NOPR's design is modeled after a recent cohort study that prospectively collected data on a continuous series of PET scans performed for all cancer-related indications at one U.S. academic medical center (4). Similar methodology has been used at other institutions to assess the impact of PET on clinicians' intended patient management plans (5–12).
Institutional Review Board Approval
The NOPR Working Group asked the ACR Institutional Review Board (IRB Federal Assurance #0002271) to determine whether the activities of the NOPR constituted research and, if so, what informed consent procedures would be required. The ACR IRB determined that data collection by the NOPR itself is not research, because it is required by CMS for reimbursement purposes for all patients having PET under the conditions of the registry. However, the IRB determined that the activities of the Working Group to analyze the data are considered research. The ACR IRB granted approval to the Working Group to engage in research using a dataset that includes data only from those patients who provide oral consent to staff at the PET facility and referring physicians who indicate consent on the form completed after the PET scan. Under Title 45 CFR
46.117(c)(2) (2005) regarding minimal risk of harm to subjects, written documentation of consent was waived by the IRB. In addition, the ACR IRB concluded that individual PET facilities and referring physicians and their staffs are not engaged in research and, therefore, are not required to have IRB review and approval before their participation in NOPR under Title 45 CFR
46.117(c)(2). The ACR IRB also determined that CMS is not conducting research. The Office of Human Research Protections separately reviewed the NOPR procedures for protection of human research subjects and found them in compliance with all applicable Department of Health and Human Services (DHHS) regulations. The DHHS also reviewed the NOPR procedures and found that they met the requirements for protection of patient confidentiality required by the Health Insurance Portability and Accountability Act.
Registry Workflow
Figure 1 illustrates the NOPR operations schematically. The responsibilities of the facility and the referring physician are outlined. Patients referred to NOPR-registered PET facilities for NOPR-eligible cancer type and indication categories are registered in the database via a secure Web-based application. Each eligible PET study receives a unique NOPR case number. For each study, the facility reports the type of scan done (PET vs. PET/CT), the specific scanner used that has already been profiled in the database, and the interpreting physician.
|
|
|
Before the PET study or when the patient arrives for the PET scan, the PET facility provides the patient with the ACR IRB-approved Patient Information Sheet (Appendix). The patient may indicate his or her consent orally to the staff at the PET facility on the day of the PET study or by telephone within 2 working days after the PET study is completed. Written consent is not required, as noted above. The PET facility records in the database whether the patient provides or withholds consent for use of his or her data in future NOPR research. If consent is withheld, data for this PET study are not included in the research database used by the Working Group.
Once the PET scan has been completed, the PET facility uploads the PET report to the database. The final step is the completion of a Post-PET form by the referring physician, who sends it to the PET facility for data entry. There are several Post-PET forms. These forms differ slightly from each other, depending on the reason for the PET study; however, all Post-PET forms assess the referring physician's planned management of the patient in light of the PET findings (Fig. 2). The physician is asked on this form for consent to use the data for NOPR research. If his or her consent is withheld, data for that PET study are not included in the research database used by the Working Group. The Post-PET form data must be entered via the Web site within 30 d after the PET study.
Case Completion and Reimbursement
Once all required NOPR forms have been entered into the database in a timely fashion, the PET facility is notified that it may submit its claim to CMS for the PET study (global claim or separate professional and technical claims). As explained earlier, the dataset compiled for use by the Working Group contains the data for those PET scans for which both the patient's and the referring physician's consent have been obtained. The entire dataset, regardless of consent decisions, is provided to CMS.
Statistical Analysis
Several prospective and retrospective studies have examined the change in patient management after a PET scan. Reported frequencies have ranged from 10% to 40% (4–12). Most of these publications were from single centers that assessed oncologic PET in aggregate or focused on common specific cancers and indications currently covered by CMS for Medicare beneficiaries.
The Working Group will evaluate the change in intended management for the currently noncovered cancers and indications for all patients as well as for each cancer type/clinical indication category individually. PET studies obtained for diagnosis of breast cancer in patients with a suspicious lesion on mammography or physical examination and those obtained for axillary nodal staging in patients with breast cancer or for regional nodal staging in patients with malignant melanoma are not included in the NOPR, because these indications are explicitly excluded by CMS (2).
To determine whether a category should be recommended to CMS for coverage, we will determine the proportion of cases in that category for which a change in management would be clinically meaningful. The "clinically important" change in management may differ when cases are assessed in aggregate as opposed to assessment for each specific cancer type/indication category. If all cases of initial staging are considered, a minimal threshold may be a 25% change in management. Yet, for particular cancers and particular indications, a clinically important effect size may differ. For example, a change in intended management in only 5% of patients with hepatocellular carcinoma being staged for planned liver transplantation might be judged as both very important and sufficiently robust to warrant requesting CMS coverage for this indication. On the other hand, a change of 30% might not be judged important if the intended management change was from first-line to second-line chemotherapy for pancreatic cancer. What constitutes a clinically important change will be determined primarily by the Working Group on the basis of available literature for the specific disease category and advice from outside experts for any disease category in which a consensus cannot be reached.
Table 2 illustrates, at different potential sample sizes, the 1-sided 95% confidence intervals (CI) for true proportions of changes in intended clinical management of 5%, 10%, 20%, 30%, and 50%, respectively. The change in intended management was modeled as a binary variable (parameter P) following a binomial distribution. The lower 95% CI bounds for the binomial proportion based on a normal approximation are shown.
|
In the second phase, we will examine the data for each cancer type/indication pairing once 200 patients have been accrued to that category or after NOPR has been operational for 1 y. The proportion of patients for whom a change in intended management is observed will be determined. If the estimated proportion of change were below the predetermined minimum clinically important change, coverage would not be recommended to CMS. If the estimated proportion were above the minimum clinically important change, the lower bound for a 1-sided 95% CI based on that proportion would then be calculated. The predetermined minimum clinically important change will then be compared with this lower 95% CI bound. If its value for the minimum clinically important change falls below the lower confidence bound, then the Working Group will recommend that CMS consider coverage outside of the NOPR for that cancer type/clinical indication category. If its value is greater than the lower confidence bound, then under the same estimated true proportion of change, the number of additional patients that must be accrued to obtain an estimate of the desired lower 95% CI bound will be computed. A decision will then be made after the additional patients have been recruited.
For example, if the estimated true proportion of change is 20% after the first 200 patients for a specific category have been accrued, then—on the basis of the numbers in Table 2—the lower bound for a 1-sided 95% CI is 15.3%. If the minimally clinically important change were judged to be 15%, then that category would be recommended to CMS for coverage because the minimum clinically important change is below the estimated lower 95% CI bound. If 17% were judged to be the minimum clinically important change, then under the assumption that the estimated true proportion of change is 20%, an additional 481 patients need to be recruited to determine that the estimated lower 95% CI bound is above 17%.
Priority Cancer and Indication Combinations
The Working Group reviewed 30 principal cancer types (e.g., pancreatic cancer) and 4 indications (diagnosis, staging, restaging/recurrence, treatment monitoring) for a total of 120 combined data cells. Table 3 lists the 15 priority areas considered by the NOPR Working Group to warrant early evaluation. Table 3 includes 2 conditions for diagnosis, 4 for staging, 4 for restaging or suspected recurrence, and 5 for therapeutic monitoring. Note that there is no overlap of cancer types between therapeutic monitoring priorities and the other 3 indications.
|
|
|
The "treatment monitoring" indication will include additional unique elements since treatment has already been initiated. Categories of change in management include adjustments in dose or duration of therapy, switching to another form of therapy, and stopping current therapy. Referring physicians are also asked "if PET were not available, would you have done some type of alternative assessment at this time?"
In secondary analyses, logistic regression models will be used to assess the impact of PET on intended management by controlling for several covariates. In these models, the dependent variable, change or no change of intended management after PET, will be modeled as a function of cancer type, cancer indication, PET facility characteristics (fixed vs. mobile), type of scan (PET vs. PET/CT), patient performance status, and referring physician as treating physician. Also, regression models for multi-category outcomes will be used in secondary analyses to further explore moderate changes in intended management. For example, a cumulative logit model for ordinal response could be applied to a 3-category treatment change: no change, intramode change, intermode change. To account for potential clustering effects in the data, such as physician or hospital effect, mixed-effect models or marginal models based on generalized estimating equations will also be considered in our future analyses.
| DISCUSSION |
|---|
|
|
|---|
There are several important limitations of the data being collected by the NOPR. The quality of the primary endpoint data is largely dependent on the care with which referring physicians or their staff members complete the Pre-PET and Post-PET forms. These forms were designed to be essentially self-explanatory, as it was not possible to provide specific training to the large number of individuals involved in this activity; accordingly, variability in the accuracy of form completion was an expected limitation of the data. To assess the magnitude of this potential problem, the Working Group has periodically evaluated samples of the data. The reviews included evaluation of the internal consistency of answers to related questions on the forms (e.g., comparing the summary stages reported before and after PET with the question asking whether PET showed the disease extent to be more, less, or the same as that judged before PET). The reviews also included comparisons of the Pre-PET and the Post-PET form responses with the somewhat more complete information in the clinical reports of the PET examinations. These reviews generally showed the data to be of reasonable quality but made some problems evident. For example, it became apparent that there was confusion about the meaning of the term "diagnosis" as a reason for ordering a PET scan, as it seemed that some respondents selected this reason when diagnosis of a recurrence of a known cancer was the intent of the examination. Confusion between "treatment monitoring" and "restaging" also was noted in some cases. These observations led us to include detailed definitions of the reasons for ordering PET scans on the Pre-PET form. We also found that some cases were being entered into the NOPR for cancers and indications that were covered by CMS; we communicated with participating PET facilities to increase their awareness of this concern. Several similar concerns were addressed by broadcast communications to participating PET facilities (these remain posted on the NOPR Web site for ready consultation). Additionally, the Working Group investigators have responded to many individual requests for clarification about subject eligibility or completion of answers to specific questions on the Pre-PET and Post-PET forms.
The NOPR will provide data for a very large number of Medicare patients and from a large fraction of all PET facilities in the United States about the changes in intended patient management after PET. It is not possible to determine from the data in the registry whether the reported changes in intention result in an actual change in patient care. The NOPR investigators hope to link the registry data with CMS billing records in the future to assess the level of agreement between the intended and actual patient management.
Even if PET does change patient management, it is unclear whether such changes will have a positive impact on patient outcomes. Although information on certain patient outcomes may be obtained from Medicare billing records, because of the large range of potential outcomes, targeted studies must be designed to assess the impact of PET on specific positive or negative patient outcomes. For example, PET may facilitate more effective treatment of disease if it detects potentially treatable cancer metastases not identified by other imaging modalities and may lead to a reduction in cancer mortality. On the other hand, PET may detect previously unsuspected incurable metastatic disease. In these situations, a shift in intended management from curative to palliative care may result in no mortality reduction but result in improved patient quality of life. Changes in patient management after PET may also result in adverse outcomes for patients, especially if the imaging has low sensitivity or specificity for certain cancers and indications or if the altered course of patient management is ineffective or potentially risky.
The NOPR differs from the hierarchic model of diagnostic efficacy described by Fryback and Thornbury (13) because it is a compendium of observational data rather than a specific study designed to answer a specific question. The NOPR may be used to document that PET changes patient management and to generate hypotheses for future research. If PET is found to have a large impact on changes in patient management in the registry, then these findings can be validated in the CMS billing data, and if the agreement between reported changes in intended management and actual management is high, then the change in patient outcomes can be assessed for specific cancer types and indications.
| CONCLUSION |
|---|
|
|
|---|
| APPENDIX |
|---|
|
|
|---|
NOPR Operations Manual (http://www.cancerpetregistry.org/pdf/nopr_opsman.pdf)
NOPR Forms (http://www.cancerpetregistry.org/forms.html)
| ACKNOWLEDGMENTS |
|---|
| FOOTNOTES |
|---|
| References |
|---|
|
|
|---|
Related articles in JNM:
This article has been cited by other articles:
![]() |
R. E. Coleman, B. E. Hillner, A. F. Shields, F. Duan, D. A. Merlino, L. G. Hanna, S. H. Stine, and B. A. Siegel PET and PET/CT Reports: Observations from the National Oncologic PET Registry J. Nucl. Med., January 1, 2010; 51(1): 158 - 163. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. S. Kibel, F. Dehdashti, M. D. Katz, A. P. Klim, R. L. Grubb, P. A. Humphrey, C. Siegel, D. Cao, F. Gao, and B. A. Siegel Prospective Study of [18F]Fluorodeoxyglucose Positron Emission Tomography/Computed Tomography for Staging of Muscle-Invasive Bladder Carcinoma J. Clin. Oncol., September 10, 2009; 27(26): 4314 - 4320. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. E. Hillner, B. A. Siegel, A. F. Shields, D. Liu, I. F. Gareen, E. Hunt, and R. E. Coleman Relationship Between Cancer Type and Impact of PET and PET/CT on Intended Management: Findings of the National Oncologic PET Registry J. Nucl. Med., December 1, 2008; 49(12): 1928 - 1935. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. N. Levine and J. A. Julian Registries That Show Efficacy: Good, but Not Good Enough J. Clin. Oncol., November 20, 2008; 26(33): 5316 - 5319. [Full Text] [PDF] |
||||
![]() |
D. L. Schwartz, H. A. Macapinlac, and R. S. Weber FDG-PET Staging of Head and Neck Cancer--Can Improved Imaging Lead to Improved Treatment? J Natl Cancer Inst, May 21, 2008; 100(10): 688 - 689. [Full Text] [PDF] |
||||
![]() |
B. E. Hillner, B. A. Siegel, D. Liu, A. F. Shields, I. F. Gareen, L. Hanna, S. H. Stine, and R. E. Coleman Impact of Positron Emission Tomography/Computed Tomography and Positron Emission Tomography (PET) Alone on Expected Management of Patients With Cancer: Initial Results From the National Oncologic PET Registry J. Clin. Oncol., May 1, 2008; 26(13): 2155 - 2161. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | RSS | TABLE OF CONTENTS |
| JOURNAL OF NUCLEAR MEDICINE TECHNOLOGY | THE JOURNAL OF NUCLEAR MEDICINE |