We are clearly on the verge of a revolution in medicine—one in which the sequence of As, Ts, Gs, and Cs is as much a part of a patient’s phenotypic profile as are the sounds of the heart or were the colors of the bile. Yet we are clearly not there yet, with many hurdles still to be overcome.
At the recent “Asia Pacific Bioinformatics” conference in Vancouver, Brad Popovic, Ph.D., CSO at Genome BC, moderated a session where stakeholders from pathologists and funders to bioinformaticians and large technology firms discussed how to traverse the technical, political, cultural, and other landscapes while avoiding the hidden mines on the way to viable medical informatics. The panel discussion helped to provide attendees—who Dr. Popovic described as largely trainees, and largely from the broadly defined bioinformatics field—with some guideposts to get from here to there.
Sequencing technology has been advancing at such a clip that in some ways it is outpacing our ability to assemble and interpret the data it is able to generate, said Dr. Popovic.
Among the biggest concerns in medicine—and in medical testing specifically—are whether data is accurate, reproducible, and translatable from one instrument (or algorithm, or lab, or technician) to another.
Jordan Stockton, Ph.D., director of product marketing, computational biology for Illumina, was asked, “How do I know what truth is? How do I know when I see a call, or a variation, or an interesting marker, that I know I’m looking at something real?”
He believes the answer has two parts. On the one hand, vendors like Illumina “own” the “accuracy problem”—of optimizing the fidelity of the calls, and alerting the end user when they should or shouldn’t trust the data. That problem, he said, is “addressable if not solved.”
(A greater challenge is in knowing what the data means, and in what context should it be taken to mean something medically serious or medically actionable, Dr. Stockton said. That is something that will potentially evolve rather than being solved in one fell swoop, and will be tackled by the community and vendors together.)
The medical community wants to see measures of confidence and portability. One way for this to happen—to move the field forward—is to coalesce around a particular set of standards. “I think in the past three years there has been a lot of consensus around various means of variant calling and various means of representation of data,” Dr. Stockton said.
He added that Illumina’s market presence allows the company the opportunity, and to some extent gives it the burden, to impose some standardization. Using a cloud-based system to analyze the data churned out by sequencers, too, “gives people the opportunity to literally be using the same code to execute the same test anywhere in the world.”
Research vs. Clinical Spheres
As of now the field is mostly in the research mode, trying to collect data, write algorithms to deal with the data, experiment with human interfaces, and debate the ethics, decide on regulation, and explore options for reimbursement.
Clinicians will need to know when a result is actionable and when it is a “variation of unknown significance (VUS),” said pathologist Aly Karsan, M.D., head of clinical diagnostic genomics and director of the cancer genetics lab at the BC Cancer Agency. It’s important that a test has been validated.
Treatment decisions require clear, crisp answers. Yet for the most part we haven’t done enough genomes to know what “normal” is, and we don’t know how extensively SNPs and indels pervade, pointed out George Michaels, Ph.D., director of life sciences programs in the Health Strategy and Solutions Group at Intel: “Interpreting genomic information in terms of the context of individual and populations really is in the early stages of directly impacting clinical practice.”
The complexity involved in personalized medicine is only beginning to be dealt with computationally. For now it’s unlikely that a genomic analysis can help recommend a particular drug or treatment—with possible exceptions in oncology—although Dr. Michaels predicts that affordable proactive personal genome technologies will soon be able to identify those that should not be used.
“There is going to have to be some larger consortia to actually define what variants actually impact on the biology of the case at hand,” Dr. Karsan remarked.
Lock It Down
While physicians need—and regulators require—clear and concise answers on which to base clinical decisions, there is an inclination among those developing tests to continually try and improve their product.
Interrogating the same samples on a routine basis should yield the same result. Even minor fixes in the software that calls bases coming off a sequencer, for example, or the algorithms into which those calls are being fed, can result in different data being delivered, and this is “not really stable enough to be stable from a clinical perspective,” said Dr. Karsan. The first step for genomic approaches to be truly useful is “locking down the informatics approaches.”
That doesn’t have to mean throwing away the key, though. “Locking down means that, on a day to day basis, you use what you have,” he noted. “If you find a bug, you fix it, but then you test and you go back and ask, ‘Does this bug-fix actually impact on previous results?’”
Algorithms can also be improved just by virtue of having a larger training set. Take a hypothetical algorithm, based on mining the data of 5,000 patients, from which were collected not just one variable-worth of data but literally millions. It uncovers a set of 22 biomarkers useful in establishing a prognosis, it’s vetted in clinical trials, it’s marketed as a test in CLIA labs, and it’s ordered by oncologists. But then fast-forward two years and 15,000 more patients, and “based on these 20,000 patient experiences and genomic data, it’s these 22 markers plus another 5 markers that we’ve found,” offered Elai Davicioni, president and CSO of GenomeDX.
Yet from both a regulatory and reimbursement perspective, a new test based on the 27 markers cannot simply be marketed as the new and improved version of the original test. A test has to be locked down, immutable: “If you want to change you basically have to go through the whole process again,” he lamented, although he does not see an easy solution to the dilemma.
Informatics software, like any black box, faces “the most rigorous bar for anything to go through FDA approval,” pointed out Dr. Popovic. “You’d better have an algorithm that can be replicated a gazillion times in different clinical environments, against different types of samples, if you’re ever going to put that into clinical medicine. I don’t think the field has matured yet to really realize that.”