Robust AI for Digital Pathology

AI-based diagnostic support in Digital Pathology

With an aging population and accompanying increase in the number of cancer cases, as well as an increasing number of complex diagnostic procedures for new therapies in cancer treatment, the workload in pathology is unceasingly increasing. At the same time, there is a shortage of specialists. Digitization together with artificial intelligence methods offer new opportunities for support in pathological diagnostics and help to close the gap in demand.

In contrast to conventional pathology, where the assessment of tissue samples is done directly by the pathologist at the microscope, in digital pathology the tissue sample is first digitized using a microscopic scanner. This paves the way for computer-assisted analysis of the digitized tissue sections using AI-based methods, e.g. for the detection and characterization of tumors or identification and quantification of specific cell types.


Heterogeneity and various questions as challenges

© Fraunhofer IIS
Section of a tissue sample that was digitized with six different scanners. The color differences between the different scanners are clearly visible.

A significant challenge in the development of AI-based diagnostic support lies in the strong heterogeneity of digital tissue sections, which results, for example, from differences in sample preparation between different clinics or from the use of tissue scanners from different manufacturers. As a result, the images sometimes vary greatly, for example in color tone, saturation or resolution.

It would also be preferable for the procedures to be easily adaptable to new problems in clinical research on the basis of only a small number of examples for which an expert decision is available. In digital pathology, large amounts of data are available, but providing expert knowledge for these data, e.g., in the form of markers in the data indicating the tissue type, as required for supervised learning approaches, is very time-consuming.

Both aspects are addressed in our research. Using the diagnosis of colorectal adenocarcinoma (colorectal cancer) as an application example, robust and adaptable AI methods for the automatic detection of tissue types (such as tumor tissue or muscle tissue) are being developed and researched in a specific manner.

Revolutionizing Digital Pathology with Few Data and Few Labels Learning Methods

The basis for the automatic segmentation of a tissue section into different tissue classes is a so-called "convolutional neural network" (CNN). This neural network is trained using sample images. To evaluate the robustness of various models for example, tissue sections were digitized with six different scanners. On these data the classification quality of the trained models is compared.

One approach to generate robust models is domain-specific data augmentation, as it is being further developed in the ADA Lovelace Center's Few Data Learning competence pillar. In data augmentation, additional images are artificially generated from existing reference images using specified transformations, such as changes in brightness or color tone. The basic idea here is to take into account the heterogeneity in the later application already during training by specifically manipulating the training images. Different augmentation techniques and their combinations are compared. The focus is on changes in the color values of the image. For example, the contrast and saturation of the images are changed, but application-specific color changes are also used.

Another focus is on the evaluation of so-called Few-Shot procedures from the Few Labels Learning competence pillar. In this method, the classes (in our case, tissue classes such as tumor) being distinguished can be subsequently adjusted based on a few annotated data without the need to retrain the neural network (competence pillar Few Label Learning). Different variants of so-called prototypical networks are used. The basic building block is a CNN as well. By this procedure now also new classes can be added subsequently. In each case, only a few examples (representatives) of the new class are needed and a class representation is calculated on the basis of these.

It turns out that a combination of the two methods is beneficial and increases the robustness of the prototypical networks on new data.

The next step: More speed for digital pathology

Two CNN architectures have been used so far (Xception and ResNet). In the further development of the project, different network architectures will now be compared with regard to their speed and robustness. Since a digital tissue section often consists of several gigapixels, it is a big challenge to realize the complete semantic segmentation of this slice under 10 min on a standard PC. Therefore, the selection of a neural network with low execution time is essential.


Cooperation between Fraunhofer IIS and Erlangen University Hospital in Digital Pathology

Privacy warning

With the click on the play button an external video from is loaded and started. Your data is possible transferred and stored to third party. Do not start the video if you disagree. Find more about the youtube privacy statement under the following link:

With our expertise in AI-based image analysis and software development for digital pathology, we are your research partner for intelligent diagnostic solutions. Our competencies include classical methods of image processing as well as the latest deep learning approaches.