IOM formed the committee after more than 30 statisticians and bioinformaticians sent a letter in July 2010 to NCI Director Harold E. Varmus, M.D., citing concerns over several genomics-based predictive tests being used in clinical trials at Duke to predict a personalized chemotherapy regimen.
Former Duke researcher Anil Potti, M.D., and colleagues published several papers claiming they could predict lung cancer progression based on gene-expression levels using microarrays. While the research and results were almost immediately questioned, they led to three clinical trials conducted at Duke in 2007 and 2008.
In detailed, if commonsense, recommendations, IOM said researchers should be assured their tests will have “a reasonable chance” of demonstrating clinical validity and utility before beginning further development. “Several criteria should be satisfied and fully disclosed (for example, through publication or patent application) to enable independent verification of the findings,” the report concluded.
Those criteria include data quality control; computational model development and cross-validation; confirmation of the computational model on an independent dataset; and release of data, code, and fully specified computational procedures to the scientific community. An independently managed database such as dbGaP should make available data and metadata used for development of omics-based tests, the report said.
The committee defined omics-based tests as including both an assay and fully specified computational procedures for analyzing assay data. The panel recommended that both be validated before use in a clinical trial. IOM also recommended that omics-based tests not be changed during clinical trials without a protocol amendment and discussion with FDA, with substantive changes possibly requiring that a study be restarted.
The report said institutions and funders can help prevent future debacles like the one at Duke by more closely watching for potential conflicts of interest, addressing questions or criticism about institutional science, and insisting that data be made available at time of publication or, for unpublished work, at end of funding. Journal editors should require lead and senior authors to attest to the integrity of the study, with the co-authors confirming shared responsibility for study integrity, while FDA should develop and finalize a risk-based guidance or regulation on bringing omics-based tests to the agency for review, over and above overseeing laboratory-developed tests.
As IOM noted, Duke allowed Dr. Potti’s work to exist outside of its established structures for reviewing and supervising clinical research, the Duke Cancer Center and the Duke Translational Medicine Institute, through the Institute for Genomic Sciences and Policy (IGSP). In finding facts and furnishing recommendations, both of which it did well, the IOM committee didn’t take the extra and necessary, albeit painful, step of answering why Duke allowed numerous entities to oversee clinical research, except for a weak explanation: “As Califf stated, the IGSP was supposed to be consultative with other research groups within the health sciences, but things got jumbled up.” Robert M. Califf, M.D., is vice chancellor for clinical and translational research at Duke University.
The multiplicity of research entities at Duke likely prevented any one from taking leadership and pursuing answers to numerous questions. Confusion existed over several points including whether researchers should disclose details to conflict-of-interest watchers (yes) and whether FDA rules requiring Investigational Device Exemption applications should apply to use of genomic predictors in clinical studies (yes).
IOM performed a valuable service in developing its guidelines. The rules, however, will prove only as valuable as the extent to which institutions follow them and especially enforce them the next time a research institution allows someone to bypass its established infrastructure and rush in to print faulty findings based on invalid data.