Abstract
Pharmaceutical companies typically perform prospective, multicenter phase 3 clinical studies to support approval of a new imaging agent by the U.S. Food and Drug Administration (FDA). In uncommon situations, the FDA has approved imaging agents based solely, or in large part, on the clinical study experience described in published reports, including reports of exploratory (i.e., phase 1 or 2) studies performed at a single clinical site. We performed a survey of published reports to assess the potential of the reported information to support FDA approval of a commonly cited investigational imaging agent. Our survey revealed critical data limitations in most publications, all of which reported exploratory clinical studies. Here we summarize the precedent for FDA approval of imaging agents using effectiveness data from publications, FDA guidance, and our experience in reviewing publications. We also present a key-data checklist for investigators to consider in the design, conduct, and reporting of exploratory clinical studies for publication. We encourage editors and peer reviewers to consider requiring these key data when reviewing these reports for publication.
The U.S. Food and Drug Administration (FDA) regulates imaging agents as drugs. Before approving a new drug, FDA law and regulations require manufacturers to verify the drug’s effectiveness in adequate and well-controlled clinical studies. Typically, prospectively designed, multicenter phase 3 clinical studies are performed to obtain the definitive effectiveness data. Although these types of studies are widely recognized as the most robust source of efficacy data, the FDA has been flexible in interpreting the nature of “adequate and well-controlled clinical studies” by describing situations in which a single adequate and well-controlled clinical study may verify a drug’s effectiveness and also the unique situation in which published reports alone may establish a new drug’s effectiveness. Indeed, the FDA has long noted that the effectiveness of a small number of new drugs was established primarily or exclusively through published reports, including secretin for evaluation of pancreatic function, bleomycin and talc for malignant pleural effusion, and doxycycline for malaria (1).
To date, 5 FDA-approved imaging drugs have had definitive effectiveness data derived from published reports (Table 1), including one drug for which site images and source data were reevaluated. The effectiveness data for 18F-FDG and 13N-ammonia injection were generated from reviews the FDA performed after implementation of the 1997 FDA Modernization Act, which contained directives specifically applicable to PET drugs (2–4). In 2012, the FDA approved 11C-choline injection using publication-based effectiveness data that pertained solely to patients with suspected prostate cancer recurrence and noninformative conventional imaging results (5). The 2016 FDA approval of 18F-fluciclovine injection (6) relied on a reinterpretation and reanalysis of source data and images from 2 single-site published clinical studies. Also in 2016, 2 published reports provided important supportive effectiveness information for the approval of 68Ga-DOTATATE injection (7). Most of these publications described single-center, exploratory (i.e., phase 1 or 2) clinical studies. Building on this precedent, we performed a pilot review of published literature to assess its potential to support FDA approval of 18F-choline for use in the setting of suspected prostate cancer recurrence. In evaluating the published reports, we relied on FDA guidance and precedent to assess the quality of the published data, as summarized below.
FDA-Approved Imaging Drugs with Effectiveness Data from Published Reports
FDA GUIDANCE ON THE USE OF PUBLISHED LITERATURE TO ESTABLISH DRUG EFFECTIVENESS
FDA guidance and public presentations by FDA staff emphasize the importance of independent substantiation of experimental clinical study results (1,8–10). Consequently, definitive effectiveness data from more than one adequate and well-controlled clinical study are usually submitted in a new-drug marketing application to the FDA. The need for independent substantiation of clinical data is due to multiple factors: the recognition that results obtained at a single clinical center may be dependent on site- or investigator-specific factors, the presence of undetected systematic biases within a single study, and the potential that chance alone will be responsible for a single study’s results. Additionally, publication bias (e.g., the tendency to publish desirable study results vs. undesirable results) may make published data unrepresentative of the true clinical experience with a drug. These factors are reflected within the FDA’s advice on the use of published reports to establish the effectiveness of a drug, as summarized below (1).
The following factors increase the possibility of reliance on published reports alone to support approval of a new product or new use:
a. Multiple studies conducted by different investigators where each of the studies clearly has an adequate design and where the findings across studies are consistent.
b. A high level of detail in the published reports, including clear and adequate descriptions of statistical plans, analytic methods (prospectively determined), and study endpoints, and a full accounting of all enrolled patients.
c. Clearly appropriate endpoints that can be objectively assessed and are not dependent on investigator judgment (e.g., overall mortality, blood pressure, or microbial eradication). Such endpoints are more readily interpreted than more subjective endpoints such as cause-specific mortality or relief of symptoms.
d. Robust results achieved by protocol-specified analyses that yield a consistent conclusion of efficacy and do not require selected post hoc analyses such as covariate adjustment, subsetting, or reduced datasets (e.g., analysis of only responders or compliant patients, or of an “eligible” or “evaluable” subset).
e. Conduct of studies by groups with properly documented operating procedures and a history of implementing such procedures effectively.
The items cited above underscore the legal requirement for a manufacturer to supply substantial evidence of a new drug’s effectiveness. The FDA has further emphasized that the interpretation of this clinical evidence, whether derived from publications or from manufacturer-sponsored clinical studies, must be considered in the context of the drug’s proposed labeling. For example, the published literature might robustly support an imaging drug’s effectiveness only within a specific group of patients, as exemplified by the approval of 11C-choline (5).
PRECEDENT FOR IMAGING DRUG APPROVAL BASED ON PUBLISHED REPORTS
The FDA approvals for 18F-FDG (2 clinical settings), 13N-ammonia, and 11C-choline were based, in large part, on effectiveness data described in publications that met certain report selection criteria, as outlined in Table 1. Similar criteria were used to assess information within the published reports supporting the effectiveness of 68Ga-DOTATATE. Published reports were important to understanding the effectiveness of 18F-fluciclovine, although the definitive data were derived from reinterpretation of clinical site images.
FDA reviews of published data, which are available on the Internet (2–7), describe the factors reviewers considered in the selection of the most useful published reports. In general, these factors focused on the ability to assess the performance of the imaging agent relative to a truth standard, such as histology (i.e., sensitivity or specificity), the applicability of the studied subjects to a clinically relevant patient population, and the details of the image interpretation process, especially the measures used to minimize interpretation bias.
As summarized in FDA reviews for the 3 imaging agents with effectiveness data based predominantly on published reports (18F-FDG (2,3), 13N-ammonia (4), and 11C-choline (5)), most selected publications described single-center clinical studies, particularly academician-sponsored clinical studies. Across all selected publications, the total sample size of the effectiveness database for these agents ranged from over one thousand subjects (18F-FDG) to 98 subjects (11C-choline). The number of subjects within any single key study report ranged from 193 (13N-ammonia) to only 13 (11C-choline). The FDA reviewers cited many limitations of the published data, including numerous deficiencies in study detail as well as the observation that some publications appeared to represent repetitive reports of the same patients. In these apparent repetitive reporting situations, the reviewers selected only the publication that contained the largest number of patients.
Published reports definitively supported the use of 18F-FDG in both a cardiac and an oncologic setting, which the FDA reviewed separately. The reviews, performed in 1999 (2,3), included extensive reports of performance data for the cardiac indication and more limited performance data for the oncologic indication. The extent of sensitivity and specificity results in the cardiac setting appeared to parallel the straightforwardness of establishing a truth standard in this setting (e.g., coronary arteriography or an independent measure of myocardial perfusion), compared with the more challenging truth standards in the oncologic setting (histopathology or other imaging modalities or clinical follow-up). Among the publication deficiencies, the 18F-FDG reviewers were especially concerned about the paucity of image interpretation information. For example, in the cardiac setting, the FDA reviewer noted, “…blinding of the readers to critical information was sufficient, though not optimal, in this set of articles.” Similarly, in the oncologic setting, the reviewer noted, “Many details about image evaluation were absent from the studies, inter- and intrareader variations in interpretation were, on the whole, either not mentioned, or discussed in limited fashion.” Nevertheless, the reviewers concluded that the totality of the selected publications, all of which described prospective clinical studies, demonstrated substantial evidence of 18F-FDG efficacy.
The FDA’s review of 13N-ammonia publications, also performed in 1999 (4), focused heavily on a key study that the FDA reviewer assessed as meeting the expectations for a prospective adequate and well-controlled clinical study, including the use of a sample size (n = 193) that allowed an estimate of imaging outcomes in patient subsets. The meaningfulness of this study’s results was bolstered by the findings in 3 published reports of retrospective studies. The reviewer concluded that 13N-ammonia effectiveness was demonstrated on the basis of consistency in results among the studies, emphasizing how the studies were performed by different investigators and included patients with an appropriate range of clinical characteristics.
The FDA’s 11C-choline review (5) was notable in that the data supporting the agent’s effectiveness were derived largely from subsets of patients described within the publications. As summarized by the FDA, the sponsor of the marketing application performed a review of publications but selected only reports that described at least 30 patients. The FDA further examined the published literature, including reports of studies that included at least 10 patients. At the conclusion of these reviews, the nature and limited extent of the published data were reflected in the drug’s labeling, in that the drug was indicated only for use among patients with suspected prostate cancer recurrence and who had noninformative results from bone scintigraphy, CT, or MRI. The effectiveness conclusion appeared to focus on the clinical importance of the imaging result in the setting of an otherwise anatomically nonlocalized cancer.
Published clinical reports provided differing roles within the FDA reviews for 68Ga-DOTATATE (7) and 18F-fluciclovine (6). The key effectiveness data for 68Ga-DOTATATE in the neuroendocrine tumor setting were derived from a prospective single-site study in which source data were made available to the FDA for inspection and analysis. Independent substantiation of these data was provided by the FDA’s assessment of a metaanalysis and systematic review performed by the drug’s marketing application sponsor. The FDA reviewers did not regard the metaanalysis as analytically sound enough to verify effectiveness; however, FDA review of individual publications culminated in a finding of sufficient support from 2 published reports of single-site, retrospective studies, which were summarized in the drug’s labeling. The effectiveness of 18F-fluciclovine among patients with recurrent prostate cancer was initially summarized in publications on 2 single-site clinical investigations. The drug’s marketing application sponsor subsequently accessed each clinical site’s original data and images and performed data reanalyses and image reinterpretation. The key 18F-fluciclovine clinical study compared imaging results to a histopathology truth standard; supportive effectiveness was provided by a single-site study that compared 18F-fluciclovine images to 11C-choline images.
After completion of the reviews, the FDA-approved drug labeling for each imaging agent reflected the extent and nature of the clinical effectiveness data, particularly with respect to the label’s indication statement, as shown below (2–7).
Fludeoxyglucose F 18 Injection:
• For the identification of left ventricular myocardium with residual glucose metabolism and reversible loss of systolic function in patients with coronary artery disease and left ventricular dysfunction, when used together with myocardial perfusion imaging.
• For assessment of abnormal glucose metabolism to assist in the evaluation of malignancy in patients with known or suspected abnormalities found by other testing modalities, or in patients with an existing diagnosis of cancer.
Ammonia N 13 Injection indication:
• For diagnostic PET imaging of the myocardium under rest or pharmacologic stress conditions to evaluate myocardial perfusion in patients with suspected or existing coronary artery disease.
Choline C 11 Injection:
• For positron emission tomography (PET) imaging of patients with suspected prostate cancer recurrence and non-informative bone scintigraphy, computerized tomography (CT) or magnetic resonance imaging. In these patients, 11C-choline PET imaging may help identify potential sites of prostate cancer recurrence for subsequent histologic confirmation. Suspected prostate recurrence is based upon elevated blood prostate specific antigen (PSA) levels following initial therapy. In clinical studies, images were produced with PET/CT coregistration.
Limitation of Use: 11C-choline PET imaging is not a replacement for histologic verification of recurrent prostate cancer.
Fluciclovine F 18 Injection:
• For positron emission tomography (PET) imaging in men with suspected prostate cancer recurrence based on elevated blood prostate specific antigen (PSA) levels following prior treatment.
Kit for the Preparation of Gallium Ga 68 Dotatate Injection:
• For use with positron emission tomography (PET) for localization of somatostatin receptor positive neuroendocrine tumors (NETs) in adult and pediatric patients.
AN EXPLORATION OF PUBLISHED REPORTS TO POTENTIALLY SUPPORT APPROVAL OF A NEW IMAGING AGENT
Following the precedent for 11C-choline, in April 2014 we surveyed published literature to estimate its potential to support FDA approval of the investigational agent 18F-choline (18F-fluoromethylcholine). Our goal was to focus on a clinical setting that directly aligned with the labeling indication for 11C-choline, that is, to assist in the evaluation of patients with suspected prostate cancer recurrence and noninformative conventional imaging results.
In a search of the National Library of Medicine’s Medline database, we identified 171 publications signaled by the key search terms (fluorocholine or F-choline or 18F-choline or fluoromethylcholine or fluoroethylcholine and prostate or prostatic or prostate cancer). A review of all the abstracts identified 33 publications (21 prospective studies and 12 retrospective studies) describing the use of 18F-choline and a reference test in the recurrent prostate cancer setting. All 33 publications were from academic institutions and cited exploratory clinical studies. Detailed review of the publications revealed a single study describing 18F-choline imaging among patients with suspected prostate cancer recurrence and noninformative conventional imaging results (11). This single-center study reported a consensus interpretation of the 18F-choline imaging results for 30 patients relative to a pathology truth standard. The study focused on the variability of imaging results over a range of imaging techniques and acquisition-initiation times. With only a single indication-applicable study identified, our survey of the published reports suggested that the information was insufficient to verify the effectiveness of 18F-choline imaging among patients with suspected prostate cancer recurrence and noninformative conventional imaging results.
We subsequently examined the published reports to identify studies that estimated the effectiveness of 18F-choline imaging among any patients with suspected recurrent prostate cancer. We screened the publications to select for studies that provided any mention of the following items: the 18F-choline dose, whether image interpretation was masked to clinical information, the image interpretation method (independent or consensus), and the start time for image acquisition. In alignment with FDA expectations for minimizing bias, we assigned the greatest value to studies that described independent interpreter results (not consensus) and interpretation of images masked to clinical information. Only one publication met all the selection criteria (12). This single-center experience in 50 patients suggested excellent 18F-choline imaging performance, yet we could identify no other similarly detailed reports. The predominant deficiencies within the publications were as follows: minimal or no mention of image interpretation methods, no or incomplete description of a truth standard or reference test, and limited or no description of whether the truth standard or reference test was applied to all or only some of the studied patients.
CONCLUSIONS AND RECOMMENDATIONS
Translation of research on investigational imaging agents into clinical practice necessitates FDA approval of the agent, clear delineation of the imaging agent’s role in medical care, and sufficient reimbursement to ensure a ready supply of the agent. The first step in this process, FDA approval, usually follows a commercial manufacturer–sponsored drug development paradigm that includes phase 1 through 3 clinical studies. In this process, manufacturers are typically ensured of patent protection and, often, a period of marketing exclusivity for their agents. These financial market incentives provide the resources that allow manufacturers to provide extensive source data and regulatory-formatted clinical study documents to support FDA approval of a new drug; in these situations, published reports typically supply supportive or ancillary information, such as safety evaluation data from an increased number of drug-exposed patients.
Several investigational agents lack patent protection and have little or no potential for clinical development by commercial manufacturers, despite extensive publication of promising imaging results by academic investigators. In these situations, the published literature may form the only logistically feasible method of verifying the investigational imaging agent’s effectiveness. We believe greater attention to methodologic details within published study reports of these agents could markedly enhance the public health value of the research by facilitating the agent’s potential for FDA approval. Even for imaging agents with commercial development potential, the addition of key details within exploratory study reports may have an important impact on phase 3 study designs and limit patient risk during development of the imaging agent.
We have developed a checklist (Appendix A) based on the FDA precedent for imaging agent approval using published study reports. The listed items are for investigators to consider when developing an exploratory study report for publication so that the data have the potential to contribute to an application for FDA approval. Report expectations for more advanced studies that definitively assess drug effectiveness and diagnostic accuracy are standardized within the CONSORT and STARD statements (13,14).
Our checklist outlines the items that we believe represent the minimal information an investigator–author should particularly consider in the design, conduct, and reporting of an exploratory clinical study for an investigational imaging agent. We encourage attention to these items even when diagnostic accuracy may relate only peripherally to the focus of the article. We also encourage professional societies to consider refining our checklist or developing additional standards for the design, conduct, and reporting of exploratory clinical studies.
The checklist emphasizes items that intuitively seem obvious, yet our experience suggests few published reports of investigational imaging agents actually contain these data elements. Conceivably, the imaging information was not described in the studies because it was regarded as not pertinent to the key points of the publication, or the study was thought to be too exploratory in nature for further use in imaging-agent development. For example, some publications focused on variations in imaging acquisition or coregistration methods in anticipation of future clinical studies. Our observation of very limited detail within published reports of exploratory clinical studies on imaging agents aligns with the experience observed in other medical fields (15,16).
We encourage authors, editors, and peer reviewers to consider the potential of all clinical studies—including exploratory clinical studies—to add to the body of data assessing an investigational imaging agent’s effectiveness and safety. These considerations are especially important for low-mass-dose imaging agents that appear unlikely to require large population safety studies for further development. Although a single-site study may explore imaging outcomes among a small number of patients and appear to provide only exploratory information, multiple similar reports may culminate in a substantial body of safety and effectiveness data.
APPENDIX A
Minimal Considerations in the Design, Conduct, and Reporting of Exploratory Clinical Trials of Investigational Imaging Agents
To help support further imaging agent development, identify or summarize…
□ Imaging agent and dose
Mass
Radiation dose
□ Study design
Prospective or retrospective
Single-center or multiple-center
□ Patient disposition
Number enrolled
Number completing all evaluations
Number with missing images or truth standard results
□ Patient characteristics
Age
Sex
Disease status (e.g., newly diagnosed disease, suspected disease, or recurrent disease)
□ Patient preparation features
Fasting or nonfasting
Drug avoidance (or not) before imaging
□ Image acquisition features
Time of acquisition onset after drug injection
Anatomic image acquisition field (e.g., “thorax through pelvis”)
□ Image interpretation methods, especially methods to minimize bias
Independent or consensus (independent is typically preferred, with results reported by the interpreter)
Number of interpreters and type (e.g., radiologists or nuclear medicine physicians)
Randomization (or not) in image presentation
Masking (or not) to clinical data or truth standard result (masking is typically preferred)
□ Truth standard or reference test
Number of patients to whom truth standard or reference test was applied
Number of patients to whom an alternative to truth standard or reference test was applied
□ Primary endpoint
Description of primary endpoint (i.e., did the study test a hypothesis or was it solely exploratory?)
Description of any prespecified statistical analytic plan
Description of results that were derived from post hoc data explorations
□ Safety
Methods of monitoring patients for safety
DISCLOSURE
The costs of publication of this article were defrayed in part by the payment of page charges. Therefore, and solely to indicate this fact, this article is hereby marked “advertisement” in accordance with 18 USC section 1734. No potential conflict of interest relevant to this article was reported.
Footnotes
Published online Jul. 21, 2016.
- © 2016 by the Society of Nuclear Medicine and Molecular Imaging, Inc.
REFERENCES
- Received for publication May 26, 2016.
- Accepted for publication June 27, 2016.
Jump to section
- Article
- Abstract
- FDA GUIDANCE ON THE USE OF PUBLISHED LITERATURE TO ESTABLISH DRUG EFFECTIVENESS
- PRECEDENT FOR IMAGING DRUG APPROVAL BASED ON PUBLISHED REPORTS
- AN EXPLORATION OF PUBLISHED REPORTS TO POTENTIALLY SUPPORT APPROVAL OF A NEW IMAGING AGENT
- CONCLUSIONS AND RECOMMENDATIONS
- APPENDIX A
- DISCLOSURE
- Footnotes
- REFERENCES
- Figures & Data
- Info & Metrics