Files
Abstract
Background: The introduction of digital pathology into clinical practice has led to the development of clinical workflows with digital images,
in connection with pathology reports. Still, most of the current work is time‑consuming manual analysis of image areas at different scales.
Links with data in the biomedical literature are rare, and a need for search based on visual similarity within whole slide images (WSIs) exists.
Objectives: The main objective of the work presented is to integrate content‑based visual retrieval with a WSI viewer in a prototype. Another
objective is to connect cases analyzed in the viewer with cases or images from the biomedical literature, including the search through visual
similarity and text. Methods: An innovative retrieval system for digital pathology is integrated with a WSI viewer, allowing to define regions of
interest (ROIs) in images as queries for finding visually similar areas in the same or other images and to zoom in/out to find structures at varying
magnification levels. The algorithms are based on a multimodal approach, exploiting both text information and content‑based image features.
Results: The retrieval system allows viewing WSIs and searching for regions that are visually similar to manually defined ROIs in various
data sources (proprietary and public datasets, e.g., scientific literature). The system was tested by pathologists, highlighting its capabilities and
suggesting ways to improve it and make it more usable in clinical practice. Conclusions: The developed system can enhance the practice of
pathologists by enabling them to use their experience and knowledge to control artificial intelligence tools for navigating repositories of images
for clinical decision support and teaching, where the comparison with visually similar cases can help to avoid misinterpretations. The system
is available as open source, allowing the scientific community to test, ideate and develop similar systems for research and clinical practice.