September 15, 2017 (Vol. 37, No. 16)
Digital Technology and Human Intuition Will Work Together to Derive Clinically Actionable Information from Whole-Slide Scans
Once the domain of science fiction, today’s facial-recognition systems easily crunch camera data using sophisticated algorithms that define general features, gender, and even mood. However, more than 50 years ago, the first forays into digital-image processing involved the analysis of cell images, not faces. In the 1960’s, Judith M.S. Prewitt and colleagues described the use of computerized imagery for the morphological analysis of cells and chromosomes.
The last decade has witnessed great strides in the digitization of traditional microscopy using whole-slide imaging. Today, this technology is quietly revolutionizing even the most complex areas of pathology. Indeed, the field of computational pathology is a moving target with varied definitions. Overall, it represents a holistic, yet mathematical means for diagnosis that incorporates multiple sources of data (e.g., pathological, radiological, clinical, and molecular data and laboratory findings) to derive clinically actionable knowledge. The field is poised to become a global game-changer. The continuing progress, regulatory issues, and future of the field was discussed at the recent Computational Pathology Symposium, held as part of the 29th European Congress of Pathology 2017 meeting in Amsterdam, the Netherlands.
Global Game-Changer
Identifying cancer subtypes and predicting response to treatment are driving forces of precision medicine for cancer diagnostics and therapeutics. Mining the details of the intricate architecture of tumor cells requires more than the naked eye of a skilled pathologist. The disciplines of radiomics and pathomics help elaborate those features and measurements. The sophisticated algorithms employed by radiomics and pathomics extracts large amounts of quantitative features from medical images and high-resolution tissue images, respectively.
“Both approaches allow us to analyze features and measurements and understand more about disease,” explains Anant Madabhushi, Ph.D., professor and director, Center for Computational Imaging and Personalized Diagnostics, Department of Biomedical Engineering, Case Western Reserve University. “For example, we improve our predictions of treatment response, survival rate, recurrence possibilities, and disease progression.”
According to Dr. Madabhushi, these relatively new fields will change existing diagnosis paradigms: “Current therapies often use drugs that stimulate the immune system, but that only works about 20% of the time. There is a huge unmet need to know who will respond, especially in the current environment for healthcare. Clearly, the costs of treatments are, unfortunately, driven by expensive drugs that may not work. Therefore, tools that predict response more accurately will improve patient outcome, as well as costs.”
Dr. Madabhushi says the ultimate goal is to do more with less: “The idea is to take routinely acquired data and maximize knowledge gleaned using computational tools. Further, this approach may soon positively impact global healthcare.
“Given the availability of internet access, digitized pathological tests may be rapidly sent to the cloud for review by skilled pathologists. Thus, for example, a slide made from the breast cancer of any woman in the world could be comprehensively analyzed by extracting subvisual information to computationally identify subtleties, allowing the identification of patients most likely to achieve a response.”
An example of a digital tissue image where computational image analysis tools have been employed to segment every individual nucleus in the stroma and epithelium. Number and spatial arrangement of these nuclei will be interrogated and correlated with disease outcome and survival. [Dr. Anant Madabhushi]
Open-Source Platform for Digital Pathology
While quantitative image analysis in digital pathology can vastly improve the speed, objectivity, and reproducibility of whole-slide analysis and biomarker interpretation, a major challenge is developing, validating, and sharing novel algorithms.
Peter Bankhead, Ph.D., senior image analyst at Philips, developed digital pathology algorithms as a postdoctoral researcher at Queen’s University, Belfast, Ireland, to support the molecular pathology research program. What resulted was an open-source platform called QuPath.
Dr. Bankhead notes that molecular-pathology research has two major difficulties: “The images are huge, and the analysis is complex. For example, a whole-slide scan of a large tissue sample could be up to 40 GB in size uncompressed, and contain millions of cells—each of which needs to be identified, classified, and quantified. I wrote QuPath to give me the tools I needed to work with this kind of data effectively.”
QuPath now encompasses many tools designed to meet the growing need for a user-friendly, extensible, open-source solution for digital pathology and whole-slide image analysis. Dr. Bankhead explains, “On one level, QuPath is designed for users who do not need to be experts in image analysis—including pathologists. QuPath provides essential features commonly needed in pathology applications (e.g., for identifying tissue, detecting cells, and making measurements). But there is also a lot of advanced functionality, such as the ability to interactively train classifiers to distinguish between different types of cells using machine learning. These steps can be combined into bespoke algorithms for a wide range of applications, complemented by extensive visualization and annotation tools.”
Ultimately, Dr. Bankhead hopes that researchers might choose to build on QuPath’s extensibility and user-friendliness, making their algorithms accessible to all. “Achieving this relies primarily on establishing an active community of users and developers,” he insists. “This community-driven model has proven enormously successful for bioimage analysis, through the widespread adoption of open-source software such as ImageJ and Fiji. I would be very happy if QuPath might contribute to a similar move toward greater openness and reproducibility in the field of digital pathology.”
Pathologist-Friendly Integration
As tests become more sophisticated and generate data more comprehensive, the resulting information explosion requires seamless integration for efficient data management, querying, and mining. No small task. “The development of a multisite pathology informatics platform must encompass many key elements,” reports Sylvia Asa, M.D., Ph.D., professor, department of laboratory medicine and pathobiology, University of Toronto, Canada, and former pathologist-in-chief at University Health Network (UHN).
She continues, “At UHN, we engaged a team of physicians, informatics personnel, and engineers, among others, to build a pathologist-friendly platform for pathology informatics. This encompasses a sophisticated laboratory information system and whole-slide imaging for histology and immunohistochemistry that is integrated with electron microscopic images and data captured from flow cytometers, etc. Further, we also integrated cytogenetics analyses and molecular diagnostics.”
Accomplishing this task within a campus setting was one aspect, but extending it to locations in a healthcare system that encompassed a large geographic area was a separate challenge. “This platform allows reporting of every specimen by the right pathologist at the right time. Integration also facilitates participation by pathologists in multidisciplinary case conferences and virtual presentations in varied locations. Ultimately, we want to ensure that the pathologist has the greatest level of access to the best data and gives the most accurate information for high-quality personalized and precision medicine.”
The electronic approach reemphasizes the critical role of pathology as the basis of diagnostic medicine, a status it secured more than a century ago. Recalling a paper written by Sir William Osler and published in a 1909 issue of the British Medical Journal, Dr. Asa offers the following quotation: “As is our pathology, so is our practice; what the pathologist thinks today, the physician does tomorrow.”
Transitioning to Diagnostics
The transition from research to the clinic is a long and winding road for diagnostic platforms. “There are many challenges that must be overcome,” explains Michael Montalto, Ph.D., executive director and head of translational pathology and biomarker technologies, translational medicine, Bristol-Myers Squibb. New diagnostic platforms usually emerge from the exploratory research setting. In the case of digital pathology-based tests, there are signs of progress. Dr. Montalto points to immune checkpoint modulators as a driving force. “These are rapidly altering the way physicians treat cancer.”
Although these tests are helpful, there are also challenges of introducing new immunohistochemistry (IHC) tests. “When IHC slides are examined, often there is reader-to-reader variability,” says Dr. Montalto. “Also, we are learning that quantifying specific types of immune cells may be more relevant than others, and it’s nearly impossible to differentiate them under a microscope. Digital pathology and computer vision will certainly help in such cases. In the past decade, this technology has made rapid progress in the automated quantitation of positive cells, providing more robust and repeatable data in clinical trials.”
Another example of progress from research to the clinic occurred this past spring. The FDA approved marketing of Philips’ IntelliSite Pathology Solution, the first whole-slide imaging system, for the primary review and interpretation of digital surgical pathology slides. The system enables pathologists to read tissue slides digitally, instead of using a microscope, to make a diagnosis. “This is a significant step forward for digital pathology,” observes Dr. Montalto. “It paves the way for automated image analysis of companion and complementary IHC diagnostic tests to enter the clinic.”
Because exploratory research in immuno-oncology continues to expand, Dr. Montalto believes the market will continue to see more of this approach. Another future improvement will include multiplexing that assesses many proteins, not just one, which is critical for immunophenotyping in the context of the tumor microenvironment.
“We are already using it extensively on the research side of things, but the model allowing it to move into diagnostics has not yet arrived,” states Dr. Montalto. “Currently, challenges include regulatory hurdles, reimbursement, and cost. One doesn’t just use a $10,000 microscope for this sort of thing. Multiplexing also needs to be simplified. It is still very technically challenging, and must be better automated for a non-Ph.D. operator. However, these are all standard market limitations for transitioning from the research to the diagnostic market.”
It’s just a matter of time until these challenges are surmounted, according to Dr. Montalto: “Multiplexed image analysis and a host of other technologies such as next-generation sequencing, gene-expression profiling, and liquid biopsies are on the horizon. All are progressing rapidly and improving to ultimately allow everyday incorporation into the diagnostic arena. This will also greatly improve patient stratification and enhance precision medicine.”
Digital pathology may advance the development of cancer drugs such as checkpoint inhibitors by processing images that inform tumor profiling, mechanisms of action, and patient selection. For example, automated image analysis algorithms are being used to augment immunohistochemistry (IHC) testing in the clinic. In this image from Bristol-Myers Squibb, PD-L1 IHC testing of formalin-fixed paraffin-embedded lung tissue shows a staining pattern that reflects non-small cell lung carcinoma.
Into the Future
With the rise of digital pathology, pathologists are beginning to perform the most of their diagnostic reviews using computer workstations instead of microscopes. “Most of the initial hurdles of these new technologies have now been overcome, and this opens up new possibilities,” notes Jesper Molin, senior research scientist, Sectra. “For example, it is now possible to review multiple slides side by side, to store and collect key images, and to search for diagnostic information in the contrast data of the image. These are all novel tools that will make pathologists smarter in the future.”
Does that mean we may soon no longer need the eyes and intuition of pathologists? Not at all. “Machine learning will be very important within the pathology domain as many researchers and companies work on systems to automate part of the work that pathologists perform,” explains Molin. “However, instead of trying to automate the diagnostic work and lose control of the diagnostic decisions, we should try to build great tools that empower pathologists based on the novel technology. Pathologists should stay in control, because once systems are fully automated, they tend to stop improving with the same pace.”
For the future, paradigm shifts will likely continue to occur in cancer care and other therapeutics as precision medicine and personalized treatments take stronger hold. Pathologists and their interactions with nonpathology departments are central to this dream of personalized medicine. It is at the desk of the pathologist where first clinical decisions regarding the patient are made. Molin philosophizes, “Technology can today not fully replace the requirement for human interactions.”
