Bildverarbeitung für die Medizin 2022

Monitoring wound healing with optical coherence tomography (OCT) imaging is a promising research field. So far, however, few data and even less manual annotations of OCT wound images are available. To address this problem, a fully unsupervised clustering method based on convolutional neural networks (CNNs) is presented. The CNN takes image patches as input and assigns them to either wound or healthy skin clusters. Network training is based on a new combination of loss functions that require information invariance and locality preservation. No expensive expert annotations are needed. Locality preservation is applied to different levels of the network and shown to improve the segmentation. Promising results are achieved with an average Dice score of 0.809 and an average rand index of 0.871 for the best performing network version.

The segmentation of vascular systems is a challenging task since their sizes and structures vary greatly so that the spatial context becomes highly important. For further clinical analysis of the vascular system it is important to create a connected vascular tree starting from the main trunk, following the tree structure up to small branches. To address these issues, we propose a new iterative segmentation model that recursively evolves a segmentation of a vasculature by following its tree structure. Our iterative CNN alternates between three steps: First, a 3D segmentation of a sub-region is performed. Second, branches that are not part of the currently analyzed branch are removed and third, subsequent sub-regions are detected. These steps are repeated until the entire vascular system is segmented. We trained, validated and tested our model on 82 CT images. We showed that, in comparison to state of the art methods, our new model generates a more accurate segmentation, resulting in an improvement of the Dice score of 7 % and a reduction of the Hausdorff distance of approximately 20 %.

Fully automatic liver segmentation is important for the planning of liver interventions and decision support. In patients with HCC, dynamic-contrast enhanced MRI is particularly relevant. Previouswork has focused on liver segmentation in the late hepatobiliary contrast phase, which may not always be available in heterogeneous data from clinical routine. In this contribution, we demonstrate the training of a convolutional neural network across contrast phases of DCEMRI, that is on par with a specialized late-phase network (mean Dice score 0.96) but in addition is more robust to other contrast phase images compared with the specialized network.

In order to facilitate early diagnosis and prevention of osteoporosis and degenerative diseases of the spine, automated opportunistic screening in routine 3D-CT scans can be implemented to assist radiologists in clinical practice. The resource limited clinical setting demands for solutions that emphasise accuracy and robustness while oftentimes being limited by computational resources. The VerSe19 and ’20 challenges aim at addressing the task of spine CT analysis but most proposed methods require multiple-stages and are computationally complex.

Unsupervised domain adaptation (UDA) methods intend to reduce the gap between source and target domains by using unlabeled target domain and labeled source domain data, however, in the medical domain, target domain data may not always be easily available, and acquiring new samples is generally timeconsuming. This restricts the development of UDA methods for new domains. In this paper, we explore the potential of UDA in a more challenging while realistic scenario where only one unlabeled target patient sample is available. We call it Few-shot Unsupervised Domain adaptation (FUDA). We first generate targetstyle images from source images and explore diverse target styles from a single target patient with Random Adaptive Instance Normalization (RAIN). Then, a segmentation network is trained in a supervised manner with the generated target images. Our experiments demonstrate that FUDA improves the segmentation performance by 0.33 of Dice score on the target domain compared with the baseline, and it also gives 0.28 of Dice score improvement in a more rigorous one-shot setting. Our code is available at https://github.com/MingxuanGu/ Few-shot-UDA.

Unsupervised anomaly detection is often attributed great promise, especially for rare conditions and fast adaptation to novel conditions or imaging techniques without the need for explicitly labeled data. However, most previous works study different methods in a constrained research setting with a limited number of common types of pathologies. Here, we want to explore a more realistic setting and target the incidental findings in a large-scale population study with 10000 participants using a recent anomaly detection approach. Despite the difficulties in selecting a proper training set in such scenarios, we were able to produce promising quantitative results and detected 31 anomalies which were not reported previously. Evaluation by a radiologist revealed remaining open challenges when it comes to the detection of less conspicuous anomalies.

Newinnovative low-cost optical coherence tomography (OCT) devices enable flexible monitoring of age-related macular degeneration (AMD) at home. In combination with current machine learning algorithms like convolutional neural networks (CNNs), assessment of AMD-related biomarkers such as pigment epithelial detachment (PED) can be supported by automatic segmentation. However, limited availability of medical image data as well as noisy ground truth does not guarantee a high generalizability of CNN models. Estimating a segmentationrelated uncertainty can be used to evaluate the confidence of the prediction. In this work, two types of uncertainties are analyzed for the segmentation of PED in home OCT image data. Epistemic and aleatoric uncertainties are determined by dropout and augmentation at test time, respectively. Evaluations are performed using pixel-wise as well as structure-wise uncertainty metrics. Results show that test-time augmentation produces both more accurate segmentations and more reliable uncertainties.

Federated Learning is the most promising way to train robust Deep Learning models for the segmentation of Covid-19-related findings in chest CTs. By learning in a decentralized fashion, heterogeneous data can be leveraged from a variety of sources and acquisition protocols whilst ensuring patient privacy. It is, however, crucial to continuously monitor the performance of the model. Yet when it comes to the segmentation of diffuse lung lesions, a quick visual inspection is not enough to assess the quality, and thorough monitoring of all network outputs by expert radiologists is not feasible. In this work, we present an array of lightweight metrics that can be calculated locally in each hospital and then aggregated for central monitoring of a federated system. Our linear model detects over 70% of low-quality segmentations on an out-of-distribution dataset and thus reliably signals a decline in model performance.

Computed Tomography Angiography is a key modality providing insights into the cerebrovascular vessel tree that are crucial for the diagnosis and treatment of ischemic strokes, in particular in cases of large vessel occlusions (LVO). Thus, the clinical workflow greatly benefits from an automated detection of patients suffering from LVOs. This work uses convolutional neural networks for case-level classification trained with elastic deformation of the vessel tree segmentation masks to artificially augment training data. Using only masks as the input to our model uniquely allows us to apply such deformations much more aggressively than one could with conventional image volumes while retaining sample realism. The neural network classifies the presence of an LVO and the affected hemisphere. In a 5-fold cross validated ablation study, we demonstrate that the use of the suggested augmentation enables us to train robust models even from few data sets. Training the EfficientNetB1 architecture on 100 data sets, the proposed augmentation scheme was able to raise the ROC AUC to 0.85 from a baseline value of 0.56 using no augmentation. The best performance was achieved using a 3D-DenseNet yielding an AUC of 0.87. The augmentation had positive impact in classification of the affected hemisphere as well, where the 3D-DenseNet reached an AUC of 0.93 on both sides.

Simultaneous localisation and categorization of objects in medical images, also referred to as medical object detection, is of high clinical relevance because diagnostic decisions often depend on rating of objects rather than e.g. pixels. For this task, the cumbersome and iterative process of method configuration constitutes a major research bottleneck. Recently, nnU-Net has tackled this challenge for the task of image segmentation with great success.

X-ray CT geometry alignment procedures commonly use phantoms with spherical markers to estimate the geometry parameters. Obtaining precise 3D positions of the markers is crucial for an accurate alignment. A typical approach utilizes a 3D version of fast radial symmetry transform for marker detection. This method works only for a given set of radii and tends to be influenced by reconstruction artifacts.With a desire for a more robust solution, a deep learningbased approach is investigated. A 3D version of the region proposal network (RPN) is implemented to determine the position and the diameter of markers in real CT measurements of alignment phantoms. The RPN incorporates a U-Netbased backbone network to capture the multi-scale information present in the volume. Experiments to determine the robustness of the network to distortionfree, limited-angle distortions, and misalignment distortions are presented. In all the three cases, all markers can be localized within the radius precision. The results show that RPN is a promising method to determine the marker positions in distorted CT volumes.

Ophthalmological imaging utilizes different imaging systems, such as color fundus, infrared, fluorescein angiography, optical coherence tomography (OCT) or OCT angiography. Multiple images with different modalities or acquisition times are often analyzed for the diagnosis of retinal diseases. Automatically aligning the vessel structures in the images by means of multi-modal registration can support the ophthalmologists in their work. Our method uses a convolutional neural network to extract features of the vessel structure in multi-modal retinal images. We jointly train a keypoint detection and description network on small patches using a classification and a cross-modal descriptor loss function and apply the network to the full image size in the test phase. Our method demonstrates the best registration performance on our and a public multi-modal dataset in comparison to competing methods.

When training deep learning models in the medical domain, one is always burdened with the task of obtaining reliable medical data annotated by experts. However, the availability of annotated data is often limited. To overcome such limitations, this paper addresses the idea of using synthetic spine X-ray data to train a deep learning model to aid in the detection of vertebrae. For this purpose, a pipeline for automatic generation of synthetic datasets comprising synthetic Xray images and their corresponding annotations is developed and evaluated. The results of these experiments show improvements in detection rates of the model when synthetic X-ray data is added to the training dataset.

In image-guided orthopedic procedures, 2D-3D registration is an essential tool to align intra-operative 2D X-ray images with pre-operative 3D CT data. Since this is a non-convex problem, appropriate initialization is important. Here,we introduceRobIn, a robust and solely intensity-based initialization method for intervention support for pelvic fractures. Key to RobIn’s robustness is the focus on the bony pelvis region in the 2D X-ray images, which is determined by automatic segmentation. First validation studies on simulated X-rays demonstrate that RobIn successfully retrieves ground truth in about 99.7% of the cases.

Pulmonary image analysis for diagnostic and interventions often relies on a canonical geometric representation of lung anatomy across a patient cohort. Bronchoscopy can benefit from simulating an appearance atlas of airway crosssections, intra-patient deformable image registration could be initialised using a shared lung atlas. The diagnosis of pneumonia, COPD and other respiratory diseases can benefit from a well defined anatomical reference space. Previous work to create lung atlases either relied on tedious and often ambiguous manual landmark correspondences and/or image features to perform deformable interpatient registration. In this work, we overcome these limitations by guiding the registration with semantic airway features that can be obtained straightforwardly with an nnUNet and dilated training labels. We demonstrate that accurate and robust registration results across patients can be achieved in few seconds leading to high agreement of small airways of later generations. Incorporating the semantic cost function improves segmentation overlap and landmark accuracy.

The disentanglement of shape and appearance is a prominent computer vision task, that has become relevant in the medical imaging domain in recent years. Medical images are often acquired in different hospitals, by different devices and using different parameters, resulting in varying intensity profiles. However, when performing population-based analysis over various datasets, e.g. from different hospitals, it is important to be able to distinguish between changes in the anatomical shapes and device-dependent intensity changes.

Reconstruction surgery of torn ligaments requires anatomically correct fixation of the graft substitute on the bone surface. Several planning methodologies have been proposed to standardise the surgical workflow by localising drill sites or defining the drill tunnel orientation. A precise drill tunnel is of high clinical relevance to prevent detrimental changes in the reconstructed ligament’s biomechanics as well as early wall breakout with the risk of damaging neurovascular structures.

In minimally invasive surgery, surgeons always rely on camera-based techniques as, unlike in open surgery, they cannot see directly into the patient’s body. Providing the necessary equipment for surgeons to become accustomed to these types of devices can be expensive and time-consuming. A much more cost-effective and versatile approach to train surgeons or prepare already trained surgeons for upcoming operations is to create surgical simulations in Virtual Reality (VR). To establish VR in the field of minimally invasive surgery, we need to consider some VR-specific limitations. Although virtual reality provides better depth perception and localization of objects compared to a conventional desktop application, it still needs to be improved to match real-life. Our approach to reduce these offsets in VR is to use different assistive visualizations. Therefore, we conducted a quantitative user study with 19 volunteers each performing 40 trials of five different visualizations. Our results indicate that two of the five visualizations (Heatmap with Isolines and Arrow Glyphs) are able to reduce the occurrent error, making training in Virtual Reality more suitable for minimally invasive surgery.

Estimation of the hand pose of a surgeon is becoming more and more important in the context of computer assisted surgery. Previous point cloud-based neural network methods for this task usually estimate offset fields to infer the 3D joint positions. Occlusions of important hand parts and inconsistencies in the point clouds, e.g. caused by uneven exposure from the depth sensor, pose a challenge to these methods. We propose to simplify the optimization problem by only estimating a weight for each point of the cloud, such that the inferred joint position is given as the weighted sum over the input points. To better capture the directional information, we define a support point set that expands the convex hull of the hand point set and enter the union of both sets as input to our network. We propose a hierarchical graph CNN, whose graph structure enables optimal information flow between the two point sets. With a mean joint error of 9.43 mm, our approach outperforms most comparable state-of-the-art methods with an average reduction by 19%, while also reducing the computational complexity.

In interventional radiology, the optimal parametrization of the X-ray image and any subsequent software processing strongly depends on the body region being imaged. These anatomy-specific parameters are combined to create customized organ programs and are necessary to obtain an optimal image quality. In today’s workflow, these programs have to be switched manually by the surgeon, which can be complex. This paper investigates a deep learning algorithm for automatic switching of organ programs in interventional X-ray machines based on the automatic detection of the imaged anatomy. We compare multiple network architectures for cardiac anatomy classification where the algorithm has to differentiate the left coronary artery, right coronary artery, and left ventricle on radiographs without contrast medium. The best-performing model achieves a micro average F1-score of 0.80. A comparison of the model performance with expert rater annotations shows promising results and recommends further clinical evaluation.

Die genaue Vermessung der Geometrie von Amputationsstümpfen ist elementar für die Herstellung einer Prothese. Konventionell wird die Geometrie mittels Gipsabdruck oder optischen 3D Scannern ermittelt. Nachteile sind hohe Kosten, Subjektivität und mögliche Ungenauigkeiten durch Patientenbewegungen. In dieser Arbeit wird daher als Alternative der Einsatz von Ultraschall zur Vermessung untersucht. Die Machbarkeit und Genauigkeit wird mit einem klinischen B-Mode Gerät geprüft. In Ergänzung wird getestet, ob dieser Ansatz auch mit A-Mode Scans einer Stiftsonde umsetzbar ist. Der Fokus liegt hierbei besonders auf der Detektion der Hautoberfläche und des Knochens eines Oberschenkelstumpfs. Die Messungen wurden an einem Stumpfphantom aus ballistischer Gelatine mit einem 3D-gedruckten Knochen durchgeführt. Die Oberfläche konnte mittels B-Mode mit einer Genauigkeit von 1,0mm gemessen und eine 3D Rekonstruktion erstellt werden. Die Messung mittels A-Mode war mit einer Genauigkeit von 0,5 cm möglich.

In-vivo x-ray microscopy (XRM) studies can help understanding the bone metabolism of living mice to investigate treatments for bone-related diseases like osteoporosis. To adhere to dose limits for living animals and avoid perturbing the cellular bone remodeling processes, knowledge of the tissue-dependent dose distribution during CT acquisition is required. In this work, a Monte Carlo (MC) simulation-based pipeline is presented, estimating the deposited energy in a realistic phantom of a mouse leg during an in-vivo acquisition. That phantom is created using a high-resolution ex-vivo XRM scan to follow the anatomy of a living animal as closely as possible. The simulation is calibrated on dosimeter measurements of the x-ray source to enforce realistic simulation conditions and avoid uncertainties due to an approximation of the present number of x-rays. Eventually, the presented simulation pipeline allows determining maximum exposure times during different scan protocols with the overall goal of in-vivo experiments with few-micrometer isotropic CT resolution.

An increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, but translational success stories in surgery are still lacking as revealed by an international poll among experts in the field of Surgical Data Science (SDS) in 2019. To come up with a roadmap for faster clinical translation and exploitation of the full potential of SDS, we conducted a 4-round Delphi process [2] involving a consortium of 50 medical and technical experts from 51 institutions. Four areas essential for moving the field forward were identified.

The robust translation of medical imaging-based models from research to real clinical settings opens new challenges. A prominent recent case is the development of models for the prediction of COVID-19 pneumonia from planar X-Ray imaging. Hundreds of models, intended for clinical use, were published within the last months.

Celiac disease is an autoimmune disorder caused by gluten that results in an inflammatory response of the small intestine.We investigated whether celiac disease can be detected using endoscopic images through a deep learning approach. The results show that additional clinical parameters can improve the classification accuracy. In this work, we distinguished between healthy tissue and Marsh III, according to the Marsh score system.We first trained a baseline network to classify endoscopic images of the small bowel into these two classes and then augmented the approach with a multimodality component that took the antibody status into account.

In this work, we construct and train a learning-based neural network pipeline for retinal vessel segmentation on fundus images and transfer the preprocessing module directly onto OCTA en face projections. Without additional fine-tuning, the transferred module retains the edge-preserving denoising functionality, as our smart network design enables zero-shot domain adaptation. Compared to a denoising network trained on OCTA data, the transferred preprocessing module is superior with regard to performance, generalization ability, and numerical stability. In addition, Frangi’s vessel segmentation of the preprocessed images outperforms the predictions using pretrained full segmentation networks. The selection of fitting Gaussian scales of the Frangi filter in the target domain can bypass the over/under-segmentation problem caused by the vessel diameter mismatch between the two domains. The contribution of this work is two-fold: we provide an exemplary evidence that smart design of network pipelines allows for flexible functional module reuse across different imaging modalities, i.e. zero-shot domain adaptation; and we discover a deep learning-based vessel segmentation pipeline for OCTA en face images which reaches an AUC score of 0.9418 without additional training.

Trauma and orthopedic surgeries that involve fluoroscopic guidance crucially depend on the acquisition of correct anatomy-specific standard projections for monitoring and evaluating the surgical result. This implies repeated acquisitions or even continuous fluoroscopy. To reduce radiation exposure and time, we propose to automate this procedure and estimate the C-arm pose update directly from a first X-ray without the need for a pre-operative computed tomography scan (CT) or additional technical equipment.

Die genaue Segmentierung großer 3D-Volumina ist eine sehr zeitaufwendige und für die Analyse und Interpretation unabdingbare Aufgabe. Die am Synchrotron gemessene Mikrotomogramme (SRμCT) zu segmentieren, ist besonders anspruchsvoll, sowohl für algorithmische Lösungen, als auch für die Experten, da sich die Daten durch geringen Kontrast, hohe räumliche Variabilität und Messartefakte auszeichnen. Am Beispiel von 3D Tomogrammen zu Biodegradationsprozessen von Knochenimplantaten untersuchten wir die Skalierung des 2D U-Nets für hochaufgelöste Graustufenvolumina unter Verwendung von drei wichtigen Modellhyperparametern (d. h. Modellbreite, -tiefe und Eingabegröße) [1].

Swallowing disorders are commonly examined using videofluoroscopy swallowing studies (VFSS). To comprehensively evaluate the swallowing process, a typical VFSS contains different patient orientations. In order to quantify the swallowing physiology, a VFSS is systematically and temporally segmented for different patient orientations. However, no fully automatic temporal segmentation tool is available. Here, we show that in general multiple deep neural networks (DNNs) are suitable for this task. We found that a variety of optimization algorithms result in generalizing DNNs. Using a systematic architectural scaling approach, we found that an efficient ResNet18 variant is sufficient to classify a full VFSS recording of about 1800 frames in less than 14 s on conventional CPUs. In the future, our findings allow a successful clinical implementation.

In the past decade, deep neural networks have gained much attention in medical imaging applications. Especially fully supervised methods have received a lot of interest as medical decision making relies on robust predictions. The ability to be more flexible and adaptive to individual anatomical differences gives them an advantage compared to unsupervised methods. However, generating high-quality labels requires expert knowledge, attention to detail and is timeconsuming. Replacing such high-quality labels with simpler annotations, such as scribbles, bounding boxes, and points, requires a network to fill in the missing information required to perform at full-supervision-like performance. This work investigates a constrained loss function integrated in a weakly supervised training of a convolutional neural network (CNN) to obtain segmentations of four different bones on lateral X-ray images of the knee. The evaluation of the different trained models with respect to the Dice coefficient shows that the proposed loss function can improve the mean Dice score across all bones from 0.262 to 0.759 by adding a size loss to the cross entropy function while using weak labels.

Image based documentation of diagnostic findings in screening colonoscopy is currently achieved by capturing single images. Nevertheless, these lack precise information about their location in the colon. Creating a panorama map of the lumen of the colon during the examination, which shows detected lesion in their context, can support the endoscopist during the documentation process. Moreover, such a panoramic map also provides information about the completeness of an examination. An important step towards such a panoramic model is a robust 3D reconstruction of the colon in a first step. Nevertheless, as colonoscopy provides only monocular image data, 3D reconstruction of the colon is challenging. Therefore, we created a 3D reconstruction pipeline, consisting of a DCNN to estimate the depth for a single video frame and the concatenation and fusion of the depth maps to a 3D model based on feature consensus. As with real colonoscopic data, ground truth information regarding the exact extension and geometry of the colon is not available,we produced a modular 3D printed phantom of the colon to evaluate the proposed reconstruction method. The phantom was examined with standard withdrawal motions using two different colonoscopes resulting in endoscopic video streams. From these sequences the3Dreconstruction was computed, and the results were aligned and compared with the ground truth obtained from CAD-blueprint of the phantoms. In all cases, the achieved quality was highly sufficient.

Training deep learning networks is very data intensive. Especially in fields with a very limited number of annotated datasets, such as diffusion MRI, it is of great importance to develop approaches that can cope with a limited amount of data. It was previously shown that transfer learning can lead to better results and more stable training in various medical applications. However, the use of off-the-shelf transfer learning tools in high angular resolution diffusion MRI is not straightforward, as such 3D approaches are commonly designed for scalar data. Here, an extension of self-supervised pretraining to diffusion MRI data is presented, and enhanced with a modality-specific procedure, where artifacts encountered in diffusion MRI need to be removed. We pretrained on publicly available data from the Human Connectome Project and evaluated the success on data from a local hospital with three modality-related experiments: segmentation of brain microstructure, detection of fiber crossings, and regression of nerve fiber spatial orientation. The results were compared against a setting without pretraining, and against classical autoencoder pretraining. We find that it is possible to achieve both improved metrics and a more stable training with the proposed diffusion MRI specific pretraining procedure.

In case of an acute ischemic stroke, rapid diagnosis and removal of the occluding thrombus (blood clot) are crucial for a successful recovery. We present an automated thrombus detection system for non-contrast computed tomography (NCCT) images to improve the clinical workflow, where NCCT is typically acquired as a first-line imaging tool to identify the type of the stroke. The system consists of a candidate detection model and a subsequent classification model. The detection model generates a volumetric heatmap from the NCCT and extracts multiple potential clot candidates, sorted by their likeliness in descending order. The classification model performs reprioritization of these candidates using graph-based deep learning methods, where the candidates are no longer considered independently, but in a global context. It was optimized to classify the candidates as clot or no clot. The candidate detection model, which also serves as the main baseline, yields a ROC AUC of 79.8%, which is improved to 85.2% by the proposed graph-based classification model.

Fast and accurate morphological classification of cells in bone marrow samples is a key step in the diagnostic workup of many disorders of the hematopoietic system such as leukemias. In spite of its long-established key position, morphological examination of bone marrow samples has been difficult to automatise, and is still mainly performed manually by trained cytologists on light microscopes. In our contribution [1], we present a neural network for classification of light microscopy images of bone marrow samples.

To support minimally-invasive intraoperative mitral valve repair, quantitative measurements from the valve can be obtained using an infra-red tracked stylus. It is desirable to view such manually measured points together with the endoscopic image for further assistance. Therefore, hand-eye calibration is required that links both coordinate systems and is a prerequisite to project the points onto the image plane. A complementary approach to this is to use a vision-based endoscopic stereo-setup to detect and triangulate points of interest, to obtain the 3D coordinates. In this paper, we aim to compare both approaches on a rigid phantom and two patient-individual silicone replica which resemble the intraoperative scenario. The preliminary results indicate that 3D landmark estimation, either labeled manually or through partly automated detection with a deep learning approach, provides more accurate triangulated depth measurements when performed with a tailored image-based method than with stylus measurements.

Mass spectrometry imaging (MSI) is a label-free analysis method for resolving biomolecules or pharmaceuticals in the spatial domain. It offers unique perspectives for the examination of entire organs or other tissue specimens. Owing to increasing capabilities of modern MSI devices, the use of 3D and multi-modal MSI becomes feasible in routine applications — resulting in hundreds of gigabytes of data.

We introduce an image-based strategy that builds on morphological cell profiling with the purpose of predicting canonical hypertrophy stimulators as proxies for pathomechanisms in cardiology. The traditional approach relies on extracting handcrafted morphological features from unlabeled cell image data in order to reason about cell biology. In this work we employ transformer networks that automatically learn features that help identify which hypertrophy stimulator has been applied on imaged cardiomyocytes. Numerical results illustrate the high predictive performance of this type of neural networks.

Early detection and analysis of calcifications in mammogram images is crucial in a breast cancer diagnosis workflow. Management of calcifications that require immediate follow-up and further analyzing its benignancy or malignancy can result in a better prognosis. Recent studies have shown that deep learning-based algorithms can learn robust representations to analyze suspicious calcifications in mammography. In this work, we demonstrate that randomly equalizing the histograms of calcification patches as a data augmentation technique can significantly improve the classification performance for analyzing suspicious calcifications. We validate our approach by using the CBIS-DDSM dataset for two classification tasks. The results on both the tasks show that the proposed methodology gains more than 1% mean accuracy and F1-score when equalizing the data with a probability of 0.4 when compared to not using histogram equalization. This is further supported by the t-tests, where we obtain a p-value of p<0.0001, thus showing the statistical significance of our approach.

In search of prognostic markers for Covid-19 disease outcome, we propose a workflow that integrates short-termchanges in longitudinal CT imaging and laboratory data with disease outcome. For longitudinal imaging data analysis, we use deformable registration and quantify the change in status (healthy, ground glas opacity and consolidation) of the lung parenchyma at a voxel level.We identify lung tissue transformed toworse (pathological) status and increasing inflammatory parameters (i.e., CRP and IL-6) to be prognostic of extended hospital stay and worsened patient outcome. We apply the methodology to compute the predictive value of these features in the first and the second Covid-19 wave.

A two-stage approach for segmentation and removal of subcutaneous tissue layers to expose the visceral region is presented. Starting from the outer skin layer, the first step is to find the boundary between the subcutaneous adipose tissue and the muscle tissue. Subsequently, the boundary between muscle and the inner visceral region is determined. Thus adipose tissue, muscle and bone structures can be segmented and removed within the abdomen, providing the viewer with an unobstructed direct volume rendering. To evaluate the procedure, segmentations of the individual compartments of subcutaneous adipose tissue and muscle tissue were compared with corresponding expert ground truth. The implied simultaneous fat and muscle segmentation (DSC>90%) used for tissue removal is also of high diagnostic value.

Bei der Schlaganfallbehandlung mittels Thrombektomie werden Blutgerinnsel mithilfe von Instrumenten wie Kathetern und Führungsdrähten aus dem Gefäßsystem entfernt. Werden diese Instrumente mit kleinen elektromagnetischen (EM) Sensoren versehen, kann man sie durch einen EM Feldgenerator (FG) im Körper lokalisieren. Mithilfe dieser Trackingdaten und präoperativen Bilddaten könnten dem Arzt während des Eingriffes zusätzliche Informationen, wie z.B. der aktuelle Abstand zum Blutgerinnsel, angezeigt werden, was eine effektivere Durchführung unterstützen würde. Die Firma Polhemus Inc. bietet mit dem Liberty TX1 einen kleinen FG an, der nah am Eingriffsort platziertwerden kann und potentiellwenige Störungen in der intraoperativen Bildaufnahme verursacht. Daneben gibt es das etablierte System Aurora von Northern Digital Inc. (NDI) mit dem Tabletop FG. Mithilfe eines standardisierten Messprotokolls wurden beide FGen unter klinischen Bedingungen auf ihre Genauigkeit und Robustheit geprüft. Der NDI Tabletop FG erreichte bei der Platzierung auf der Patientenliege eine Positionsgenauigkeit von 0,38mm und einen mittleren Jitter von 0,02 mm, der Polhemus TX1 FG eine Genauigkeit von 0,24mm und einen mittleren Jitter von 0,07 mm. Außerdem wurde mithilfe eines open-science Gefäßphantoms eine katheterbasierte Intervention nachgestellt, um den Einfluss der FGen auf die digitale Subtraktionsangiographie (DSA) zu untersuchen. Der TX1 FG erzeugte durch die geringe Baugröße im Gegensatz zum Tabletop FG kaum Artefakte. Aufgrund der Kompaktheit, den tendenziell besseren Genauigkeitsergebnissen und der hohen Robustheit stellt der TX1 FG für das Instrumententracking eine interessante Alternative dar. Derzeit hat das Polhemus System allerdings noch die Einschränkung, dass dessen kleinster Sensor mit einem Durchmesser von 1,8mm minimal größer ist als der maximale Innendurchmesser eines bei der Thrombektomie eingesetzten Aspirationskatheters von 1,77 mm.

In recent years, neural networks drastically gained on popularity in the medical image processing domain since they proved to be suitable to reliably solve many complex tasks. However, neural networks typically require a large amount of ground truth annotated training images to deliver accurate results. Yet, the annotation of medical images is a very time-consuming process and mostly requires expert knowledge.

Cardiac magnetic resonance (CMR) images are widely used for cardiac diagnosis and ventricular assessment. Extracting specific landmarks like the right ventricular insertion points is of importance for spatial alignment and 3D modelling. The automatic detection of such landmarks has been tackled by multiple groups using Deep Learning, but relatively little attention has been paid to the failure cases of evaluation metrics in this field. In this work, we extended the public ACDC dataset with additional labels of the right ventricular insertion points and compare different variants of a heatmap-based landmark detection pipeline. In this comparison, we demonstrate very likely pitfalls of apparently simple detection and localisation metrics which highlights the importance of a clear detection strategy and the definition of an upper-limit for localisation based metrics. Our preliminary results indicate that a combination of different metrics are necessary, as they yield different winners for method comparison. Additionally, they highlight the need of a comprehensive metric description and evaluation standardisation, especially for the error cases where no metrics could be computed or where no lower/upper boundary of a metric exists. Code and labels: https://github.com/Cardio-AI/rvip_landmark_detection

This study evaluates the diagnostic performance for binary abnormality classification of deep learning models on various types of sequences from a multidisease clinical brain MRI dataset. Additionally, it determines the influence of the sample size and the type of disease. The sequences are DWI, FLAIR, T1- weighted, T1-weighted FLAIR, T2-weighted and T2-weighted FLAIR. On the full-sized multi-disease, the best performance is achieved on the T2-weighted FLAIR sequence using a VGG-16 dataset resulting in an AUC value of 0.89. The work highlights the importance of carefully selecting MRI sequences for deep learning and identifies discrepancies to screening protocols for physicians.

The onset of rheumatic diseases such as rheumatoid arthritis is typically subclinical, which results in challenging early detection of the disease. However, characteristic changes in the anatomy can be detected using imaging techniques such as MRI or CT. Modern imaging techniques such as chemical exchange saturation transfer (CEST) MRI drive the hope to improve early detection even further through the imaging of metabolites in the body. To image small structures in the joints of patients, typically one of the first regions where changes due to the disease occur, a high resolution for the CEST MR imaging is necessary. Currently, however, CEST MR suffers from an inherently low resolution due to the underlying physical constraints of the acquisition. In this work we compared established up-sampling techniques to neural network-based super-resolution approaches. We could show, that neural networks are able to learn the mapping from low-resolution to high-resolution unsaturated CEST images considerably better than present methods. On the test set a PSNR of 32.29 dB (+10%), a NRMSE of 0.14 (+28%), and a SSIM of 0.85 (+15%) could be achieved using a ResNet neural network, improving the baseline considerably. This work paves the way for the prospective investigation of neural networks for super-resolution CEST MRI and, followingly, might lead to a earlier detection of the onset of rheumatic diseases.

Efficient and fully automatic multi-organ segmentation is of great research and clinical prospect. Deep learning (DL) based methods have recently emerged and proven its effectiveness in various biomedical segmentation tasks. The performance of DL based segmentation models strongly depends on the training dataset and a large, correctly annotated dataset is always crucial. However, gathering annotation for multi-organ segmentation task is difficult and making use of public datasets with existing annotations then becomes one possible solution. In this work we propose a pipeline for training multi-organ segmentation model from partially annotated datasets. The proposed method is evaluated using left, right lungs and liver segmentation task of throat-abdomen CT scans. From average dice score, we found the proposed method can obtain very close performance using only partially annotated datasets (0.93), compared with models using fully annotated datasets (0.96).

Cellular morphology has the capacity to serve as a surrogate for cellular state and functionality. However, primary cardiomyocytes, the standard model in cardiovascular research, are highly heterogeneous cells and thus impose methodological challenges to analysis. Hence, we aimed to devise a robust methodology to deconvolute cardiomyocyte morphology on a single-cell level: C-MORE (cellular morphology recognition) is a workflow from bench to data analysis tailored for heterogeneous primary cells using our R package cmoRe.

Annotating data, especially in the medical domain, requires expert knowledge and a lot of effort. This limits the amount and/or usefulness of available medical data sets for experimentation. Therefore, developing strategies to increase the number of annotations while lowering the needed domain knowledge is of interest. A possible strategy is the use of gamification, i.e. transforming the annotation task into a game. We propose an approach to gamify the task of annotating lung fluid cells from pathological whole slide images (WSIs). As the domain is unknown to non-expert annotators, we transform images of cells to the domain of flower images using a CycleGAN architecture. In this more assessable domain, non-expert annotators can be (t)asked to annotate different kinds of flowers in a playful setting. In order to provide a proof of concept, this work shows that the domain transfer is possible by evaluating an image classification network trained on real cell images and tested on the cell images generated by the CycleGAN network (reconstructed cell images) as well as real cell images. The classification network reaches an average accuracy of 94.73% on the original lung fluid cells and 95.25% on the transformed lung fluid cells, respectively. Our study lays the foundation for future research on gamification using CycleGANs.

The proceedings of the workshops Bildverarbeitung für die Medizin are published in a unified form electronically and as bound proceedings. LATEX serves as the base for both types of publication. The template of this PDF can be used as a template, all files can be obtained from the pages of the workshop. In order to be able to guarantee a unified appearance and a smooth process, we ask you to comply with the specifications described here. If necessary, submission in MS Word format is possible, however, and extra fee is charged for this.

Image-guided diagnostics with AI assistance, e.g. compression-ultrasound for detecting deep vein thrombosis, requires stable, robust and real-time capable analysis algorithms that best support the user. When using anatomical segmentations for user guidance the spatiotemporal consistency is of great importance, but point-of-care modalities deliver signal which in many frames is hard to interpret. Since 2D+t models with 3D CNNs are not applicable for many mobile end devices,we propose a newspatiotemporal attention approach that re-uses deep backbone features from previous frames to learn and optimally fuse all available image information. Proof-of-concept experiments demonstrate an improvement of over 8% for the segmentation compared to simpler 2D+t models (using several frames as multi-channel input).

Cerebral aneurysms represent an essential problem in neuroradiology. In clinical practice, they are frequently diagnosed and treated based on digital subtraction angiography (DSA) which provides an impression of the blood flow dynamics. In contrast, computational hemodynamics enables precise quantification of flow-related properties based on a patient-specific 3D anatomy extracted from CT or MRI. To support the qualitative interpretation of simulated flow data, we imitate the appearance of DSA data from simulated flow data. This research is motivated by shortcomings of previous visualization techniques which may overwhelm physicians and are not familiar to them. The virtual DSA representations can be generated without manual parameter adaption by the user. We applied our method to different cerebral aneurysm data sets and performed a qualitative evaluation compared to real DSA images together with two radiologists.

Image reconstruction in Magnetic Particle Imaging is mainly performed by using a system matrix or by mapping the time signal into spatial domain and deconvolving the tracer properties. In this work, a neural network is designed and trained for reconstructing 1D images. Test data are reconstructed using both the neural network and a conventional approach. Background artefacts that appear during conventional reconstruction are not visible when reconstructing with the neural network. The images that have been reconstructed using the neural network are superior in terms of quantifiability and spatial resolution in comparison to conventionally reconstructed images.

In endovascular aneurysm repair (EVAR) procedures, the stent graft navigation and implantation is currently performed under a two-dimensional (2D) imaging-based guidance requiring X-rays and contrast agent. In [1], a novel three-dimensional (3D) stent graft guidance approach based on tracking systems is introduced. The method is based on a 3D guidance method which combines fiber optical shape sensing with electromagnetic tracking to obtain the 3D shape [2] of the tracked instrument, e.g., a stent graft system. In this work, the approach is extended to provide also the 3D stent graft shape.

Supervised deep learning has shown state-of-the-art performance for medical image segmentation across different applications, including histopathology and cancer research; however, the manual annotation of such data is extremely laborious. In this work, we explore the use of superpixel approaches to compute a pre-segmentation of HER2 stained images for breast cancer diagnosis that facilitates faster manual annotation and correction in a second step. Four methods are compared: standard simple linear iterative clustering (SLIC) as a baseline, a domain adapted SLIC, and superpixels based on feature embeddings of a pretrained ResNet-50 and a denoising autoencoder. To tackle oversegmentation, we propose to hierarchically merge superpixels, based on their content in the respective feature space. When evaluating the approaches on fully manually annotated images, we observe that the autoencoder-based superpixels achieve a 23% increase in boundary F1 score compared to the baseline SLIC superpixels. Furthermore, the boundary F1 score increases by 73% when hierarchical clustering is applied on the adapted SLIC and the autoencoder-based superpixels. These evaluations show encouraging first results for a pre-segmentation for efficient manual refinement without the need for an initial set of annotated training data.

Shortage of annotated data is one of the greatest bottlenecks related to biomedical image analysis in general, and surgical data science (SDS) in particular. Meta learning studies howlearning systems can increase in efficiency through experience and could thus evolve as an important concept to overcome data sparsity. A core capability of meta learningbased approaches is the identification of similar previous tasks given a new task. We recently addressed this problem and presented the concept of task fingerprinting [1], which involves representing a task (comprising images and labels), by a vector of fixed length irrespective of data set size, types of labels or specific resolutions (Fig. 1).

Chronic wounds including diabetic and arterial/venous insufficiency injuries have become a major burden for healthcare systems worldwide. Demographic changes suggest that wound care will play an even bigger role in the coming decades. Predicting and monitoring response to therapy in wound care is currently largely based on visual inspection with little information on the underlying tissue. Thus, there is an urgent unmet need for innovative approaches that facilitate personalized diagnostics and treatments at the point-of-care. It has been recently shown that ultrasound imaging can monitor response to therapy in wound care, but this work required onerous manual image annotations. In this study we present initial results of a deep learning-based automatic segmentation of cross-sectional wound size in ultrasound images and identify requirements and challenges for future research on this application. Evaluation of the segmentation results underscores the potential of the proposed deep learning approach to complement non-invasive imaging with Dice scores of 0.34 (U-Net, FCN) and 0.27 (ResNet-U-Net) but also highlights the need for improving robustness further.We conclude that deep learning-supported analysis of non-invasive ultrasound images is a promising area of research to automatically extract cross-sectional wound size and depth information with potential value in monitoring response to therapy.

Vascular malformations (VMs) are a rare condition. They can be categorized into high-flow and low-flow VMs, which is a challenging task for radiologists. In this work, a very heterogeneous set of MRI images with only rough annotations are used for classification with a convolutional neural network. The main focus is to describe the challenging data set and strategies to deal with such data in terms of preprocessing, annotation usage and choice of the network architecture. We achieved a classification result of 89.47% F1-score with a 3D ResNet 18.

With the introduction of whole slide imaging (WSI) systems, several digital pathology applications have emerged. Despite all benefits, lacking appropriate infrastructure to process proprietary WSI file formats for remote diagnosis and annotation is a constraint for widespread application of digital pathology. The joint imaging platform (JIP) already includes a wide range of solutions for digital medical image processing, mainly focused on radiology. We extend the infrastructure in the JIP for accessing, storage, remote analysis and deep learningbased processing of pathological data. By converting proprietaryWSI file formats into the DICOM standard, we enable the linkage of radiology and pathology on the JIP and show potential applications in current research studies.

The adaptation and application of medical image analysis algorithms inside a clinical environment comes with the challenge of defining, curating and annotating suitable training and testing cohorts from increasing numbers of available DICOM images. Systems for automated image retrieval and cohort selection have emerged in recent years. Commonly, however, a physician still needs to verify results and take a look at the images themselves to take a final decision. In this work, in order to assist this process and to provide functionalities for standard-conform tagging and adding of free-text to DICOM images, we combine two open source tools, namely Doccano and OHIF Medical Imaging Viewer. We integrate them into the Kaapana open source platform and imaging toolkit. We demonstrate how these functionalities can be leveraged to curate cohorts, add, adjust and enrich DICOM metadata and to tag images for image classification or image-text correlation tasks. Having these steps integrated in a DICOM-conform way also represents an important step towards adopting FAIR-principles in the scientific process.

Neuromuscular diseases (NMDs) cause a significant burden for both healthcare systems and society. They can lead to severe progressive muscle weakness, muscle degeneration, contracture, deformity and progressive disability. The NMDs evaluated in this study often manifest in early childhood. As subtypes of disease, e.g. Duchenne muscular dystropy (DMD) and spinal muscular atrophy (SMA), are difficult to differentiate at the beginning and worsen quickly, fast and reliable differential diagnosis is crucial. Photoacoustic and ultrasound imaging has shown great potential to visualize and quantify the extent of different diseases. The addition of automatic classification of such image data could further improve standard diagnostic procedures.We compare deep learning-based 2-class and 3-class classifiers based on VGG16 for differentiating healthy from diseased muscular tissue. This work shows promising results with high accuracies above 0.86 for the 3-class problem and can be used as a proof of concept for future approaches for earlier diagnosis and therapeutic monitoring of NMDs.

Semi-supervised learning offers great potential for medical image analysis, as it reduces the annotation burden for clinicians. In this work, we apply the state-of-the-art method FixMatch to chest X-ray and retinal image datasets. Our comparison with the supervised-only method is based on a fair hyperparameter tuning budget and includes label imbalance in the labeled set, thus simulating a practical evaluation setup. We find that unlabeled data can be used effectively for the retinal images, especially when using additional methods to counteract label imbalance in the unsupervised loss. In experiments with CheXpert, however, FixMatch does not provide substantial gains.

Following the deep brain stimulation (DBS) surgery, the stimulation parameters are manually tuned to reduce symptoms. This procedure can be timeconsuming, especially with directional leads. We propose an automated methodology to initialise contact configurations using imaging techniques. The goal is to maximise the electric field on the target while minimising the spillover, and the electric field on regions of avoidance. By superposing pre-computed electric fields, we solve the optimisation problem in less than a minute, much more efficient compared to finite element methods. Our method offers a robust and rapid solution, and it is expected to considerably reduce the time required for manual parameter tuning.

Die Interpretation von Thorax-Röntgenbildern ist ein zentraler, nicht-trivialer Bestandteil der Diagnose von Lungen- oder Herzkrankheiten. Bei der Erzeugung der Label, die zum Einsatz von überwachten Deep Learning Methoden notwendig sind, herrscht eine gewisse Unsicherheit, die in den Labeln festgehalten werden kann. In dieser Arbeit soll die pathologiespezifische Behandlung dieser Labelunsicherheit im Vordergrund stehen. Es konnte gezeigt werden, dass dieses Vorgehen den durchschnittlichen ROC-AUC bei mehreren Modellarchitekturen verbessert.

To restore blood flow after an ischemic stroke due to large vessel occlusion, thrombectomy is a common treatment method. The postoperative noncontrast computed tomography (CT) often shows hyperdense regions which are due to hemorrhagic transformation or contrast staining. Since further treatment decisions depend on the presence of hyperdensities, their reliable detection is necessary. The Deep Learning based approach presented in this study can support radiologists in this task. The dataset consists of 241 postoperative volumetric noncontrast CTs. They are labeled by a binary classification regarding the presence or absence of a hyperdensity. A shallow 3D CNN architecture and a preprocessing pipeline were proposed. A part of the preprocessing is windowing the CTs to enhance contrast. Different windowing thresholds were defined based on knowledge regarding the CT values of brain tissue and hyperdensities. Using the proposed window with a level of 50 HU and a width of 60 HU, the network achieved an accuracy of 89% on the test data. Without windowing, an accuracy of only 46%was achieved. The present study demonstrates the importance of appropriate preprocessing and how domain knowledge can be included to optimize it. The results indicate that reducing the input information in a meaningful way accentuates relevant features in the images and enhances the network performance.

Rupture risk analysis of intracranial aneurysms is important for treatment decisions. Morphological parameters like size, diameter or aspect ratio are used to capture the relevant aspects of the aneurysm shape and predict the rupture of intracranial aneurysms. Automatic calculation of these parameters is cumbersome, whereas manual measurements are time-consuming, error-prone and subject to inter-observer variance. Instead of classification based on morphological parameters, here, deep learning on aneurysm surface meshes is used to classify 3D surface meshes of intracranial aneurysm into ruptured and unruptured. We compared several deep learning approaches on surfaces meshes and point clouds showing patient-specific aneurysm geometries. Using 150 aneurysms for training and 40 for testing, a test accuracy of 82,5% was achieved.

Mammography is an important part of breast cancer diagnostics, as it allows to inspect the inner breast structure without physically penetrating breast tissue. Commonly, mammograms tend to vary in their visual appearance based on the specific device and the circumstances under which the mammogram is acquired. Such images could cause artificial intelligence algorithms to fail as they can introduce an undesired variation into the data. This study intends to put these images to use by utilizing a cycle-consistent Generative Adversarial Network (GAN) in order to augment the training data by diversifying instances of each visual domain into all the available ones. A publicly available dataset was augmented to train a detection network; the GANs used for the augmentation were trained with an in-house dataset with three visually different domains.Results show that using our augmentation technique consistently increases the detection performance by reaching a mean average precision of up to 0.82 against 0.77 without augmenting the data.

Fovea localization is one of the most popular tasks in ophthalmic medical image analysis, where the coordinates of the center point of the macula lutea, i.e. fovea centralis, should be calculated based on color fundus images. In this work, we treat the localization problem as a classification task, where the coordinates of the x- and y-axis are considered as the target classes. Moreover, the combination of the softmax activation function and the cross entropy loss function is modified to its multiscale variation to encourage the predicted coordinates to be located closely to the ground-truths. Based on color fundus photography images, we empirically show that the proposed multiscale softmax cross entropy yields better performance than the vanilla version and than the mean squared error loss with sigmoid activation, which provides a novel approach for coordinate regression.

X-ray microscopy (XRM) allows the investigation of osteocyte lacunae and recently discovered trans-cortical vessels in murine tibia bones due to higher resolution than conventional micro-CT (μCT) approaches. However, segmentation methods for XRM data are not yet established. Here, we propose a deep learning approach utilizing a U-Net-based neural network trained on a similar modality – μCT – that is capable of segmenting in both domains. We altered the XRM data to more closely resemble the μCT data to allow segmentation in the shifted XRM domain. Segmentation error on μCT data was evaluated by the F1 score (0.954) and IoU (0.913), whereas the segmentation on XRM data was verified visually. We conclude that the obtained model indeed allows the segmentation of cortical bone in both XRM and μCT data, although it was only trained on μCT images.

Bei endoskopischen Computer-Vision-Anwendungen sind Echtzeitverarbeitung der Videodaten sowie geringe Latenzen für einen praktischen klinischen Einsatz maßgeblich. Gleichzeitig führt die kontinuierliche Hardwareweiterentwicklung zu einer stetigen Verbesserung der Bildauflösung. Eingangsbilddaten hoher Auflösung erfordern i. d. R. eine patchweise Verarbeitung, wobei durch Patch-Priorisierungsstrategien die Verarbeitung der Daten beschleunigt und Latenzen reduziert werden können. Mit der Bildsegmentierung als Beispielaufgabe wird im vorliegenden Beitrag untersucht, wie das Patch-Sampling zur Inferenzzeit als Reinforcement Learning (RL)-Problem formuliert warden kann. Anhand von synthetischen und realen Daten wird gezeigt, dass durch das entwickelte RL-basierte Patch-Priorisierungsmodell (PPM) eine beschleunigte Segmentierung relevanter Bildregionen realisiert werden kann.

In the domain of medical image processing, medical device manufacturers protect their intellectual property in many cases by shipping only compiled software, i.e. binary code which can be executed but is difficult to be understood by a potential attacker. In this paper, we investigate how well this procedure is able to protect image processing algorithms. In particular, we investigate whether the computation of mono-energetic images and iodine maps from dual energy CT data can be reverse-engineered by machine learning methods. Our results indicate that both can be approximated using only one single slice image as training data at a very high accuracy with structural similarity greater than 0.98 in all investigated cases.

Face detection in videos from smart cars or homes is becoming increasingly important in human-computer interaction, emotion recognition, gender and age identification, driving assistance, and vital sign measurements, as heart rate and respiratory rate is derived from the video. However, face detection suffers from variations in illumination, subject motion, different skin colors, or camera distances. In this work [1], we compare three algorithms for in-car application: Haar cascade classifier (HCC), histogram of oriented gradients (HoG), and a deep neural network (DNN). For evaluation, we consider the freely available “driver monitoring dataset” (DMD) multimodal database and selfcollected videos recorded in a research car. We analyze run-time, accuracy, and F1-score. HoG has highest computational time as compared to HCC and DNN with 2.99 frames per second (FPS), 7.00 FPS, and 18.25 FPS, respectively. For DMD, the F1-scores are 91.75%, 95.91%, and 99.48% for HCC, HoG, and DNN respectively, and 88.05%, 83.68%, and 99.66% for our dataset, respectively.