Institute of Fundamental Technological Research

Monitoring the healing progress of diabetic foot ulcers is a challenging process. Accurate segmentation of foot ulcers can help podiatrists to quantitatively measure the size of wound regions to assist prediction of healing status. The main challenge in this field is the lack of publicly available manual delineation, which can be time consuming and laborious. Recently, methods based on deep learning have shown excellent results in automatic segmentation of medical images, however, they require large-scale datasets for training, and there is limited consensus on which methods perform the best. The 2022 Diabetic Foot Ulcers segmentation challenge was held in conjunction with the 2022 International Conference on Medical Image Computing and Computer Assisted Intervention, which sought to address these issues and stimulate progress in this research domain. A training set of 2000 images exhibiting diabetic foot ulcers was released with corresponding segmentation ground truth masks. Of the 72 (approved) requests from 47 countries, 26 teams used this data to develop fully automated systems to predict the true segmentation masks on a test set of 2000 images, with the corresponding ground truth segmentation masks kept private. Predictions from participating teams were scored and ranked according to their average Dice similarity coefficient of the ground truth masks and prediction masks. The winning team achieved a Dice of 0.7287 for diabetic foot ulcer segmentation. This challenge has now entered a live leaderboard stage where it serves as a challenging benchmark for diabetic foot ulcer segmentation.

Deep learning, Diabetic foot ulcers, Segmentation, Convolutional neural networks

In this study, we introduce ‘‘instance loss functions’’, a new family of loss functions designed to enhance the
training of neural networks in the instance-level segmentation and detection of objects in biomedical image
data, particularly those of varied numbers and sizes. Intended to be utilized conjointly with traditional loss
functions, these proposed functions, prioritize object instances over pixel-by-pixel comparisons. The specific
functions, the instance segmentation loss (instance), the instance center loss (center), the false instance rate
loss (false), and the instance proximity loss (proximity), serve distinct purposes. Specifically, instance improves
instance-wise segmentation quality, center enhances segmentation quality of small instances, false minimizes
the rate of false and missed detections across varied instance sizes, and proximity improves detection quality
by pulling predicted instances towards the ground truth instances. Through the task of segmenting white
matter hyperintensities (WMH) in brain MRI, we benchmarked our proposed instance loss functions, both
individually and in combination via an ensemble inference models approach, against traditional pixel-level
loss functions. Data were sourced from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) and the
WMH Segmentation Challenge datasets, which exhibit significant variation in WMH instance sizes. Empirical
evaluations demonstrate that combining two instance-level loss functions through ensemble inference models
outperforms models using other loss function on both the ADNI and WMH Segmentation Challenge datasets for
the segmentation and detection of WMH instances. Further, applying these functions to the segmentation of
nuclei in histopathology images demonstrated their effectiveness and generalizability beyond WMH, improving
performance even in contexts with less severe instance imbalance.

Instance-level segmentation loss, Instance-level detection loss, White matter hyperintensities, Brain lesions

Pairwise image registration is a necessary prerequisite for brain image comparison and data integration in neuroscience and radiology. In this work, we explore the efficacy of implicit neural representations (INRs) in improving the performance of brain image registration in magnetic resonance imaging. In this setting, INRs serve as a continuous and coordinate based approximation of the deformation field obtained through a multi-layer perceptron. Previous research has demonstrated that sinusoidal representation networks (SIRENs) surpass ReLU models in performance. In this study, we first broaden the range of activation functions to further investigate the registration performance of implicit networks equipped with activation functions that exhibit diverse oscillatory properties. Specifically, in addition to the SIRENs and ReLU, we evaluate activation functions based on snake, sine+, chirp and Morlet wavelet functions. Second, we conduct experiments to relate the hyper-parameters of the models to registration performance. Third, we propose and assess various techniques, including cycle consistency loss, ensembles and cascades of implicit networks, as well as a combined image fusion and registration objective, to enhance the performance of implicit registration networks beyond the standard approach. The investigated implicit methods are compared to the VoxelMorph convolutional neural network and to the symmetric image normalization (SyN) registration algorithm from the Advanced Normalization Tools (ANTs). Our findings not only highlight the remarkable capabilities of implicit networks in addressing pairwise image registration challenges, but also showcase their potential as a powerful and versatile off-the-shelf tool in the fields of neuroscience and radiology.

Purpose: BUS-Set is a reproducible benchmark for breast ultrasound (BUS) lesion segmentation, comprising of publicly available images with the aim of improving future comparisons between machine learning models within the field of BUS. Method: Four publicly available datasets were compiled creating an overall set of 1154 BUS images, from five different scanner types. Full dataset details have been provided, which include clinical labels and detailed annotations. Further- more, nine state-of -the-art deep learning architectures were selected to form the initial benchmark segmentation result, tested using five-fold cross-validation and MANOVA/ANOVA with Tukey statistical significance test with a threshold of 0.01. Additional evaluation of these architectures was conducted, exploring possible training bias, and lesion size and type effects. Results: Of the nine state-of -the-art benchmarked architectures, Mask R-CNN obtained the highest overall results, with the following mean metric scores: Dice score of 0.851, intersection over union of 0.786 and pixel accuracy of 0.975. MANOVA/ANOVA and Tukey test results showed Mask R-CNN to be statistically significant better compared to all other benchmarked models with a p-value > 0.01. Moreover, Mask R-CNN achieved the highest mean Dice score of 0.839 on an additional 16 image dataset, that contained multiple lesions per image. Further analysis on regions of interest was conducted, assessing Hamming distance, depth-to-width ratio (DWR), circularity, and elongation, which showed that the Mask R-CNN’s segmentations maintained the most morphological fea-tures with correlation coefficients of 0.888, 0.532, 0.876 for DWR, circularity, and elongation, respectively. Based on the correlation coefficients, statistical test indicated that Mask R-CNN was only significantly different to Sk-U-Net.Conclusions: BUS-Set is a fully reproducible benchmark for BUS lesion seg-mentation obtained through the use of public datasets and GitHub. Of the state-of -the-art convolution neural network (CNN)-based architectures, Mask R-CNN achieved the highest performance overall, further analysis indicated that a training bias may have occurred due to the lesion size variation in the dataset. All dataset and architecture details are available at GitHub: https://github.com/corcor27/BUS-Set, which allows for a fully reproducible benchmark.

breast segmentation,public datasets

Introduction: Nonalcoholic fatty liver disease (NAFLD) is a common liver abnormality, but its non-invasive diagnosis in patients with severe obesity remains difficult.
Objectives: To investigate the usefulness of the ultrasound (US) based hepatorenal index (HRI) technique, and two biomarker-based methods, including the hepatic steatosis index (HSI) and NAFLD logit score for the diagnosis of NAFLD in subjects referred for the bariatric surgery.
Patients and methods: 162 subjects, 106 with NAFLD, admitted for the bariatric surgery participated in the study. Fat fraction level and the presence of NAFLD were determined using surgical liver biopsy. Each patient underwent liver US examination and blood tests to determine the HRI, HSI and NAFLD logit score.
Results: For the NAFLD diagnosis, the HRI, HSI and NAFLD logit score techniques achieved areas under the receiver operating characteristic curves of 0.879, 0.577 and 0.825, respectively. The Spearman’s correlation coefficients between the liver fat fraction values and the HRI, HSI and NAFLD logit score were equal to 0.695, 0.215 and 0.595, respectively. The optimal cut-off values for the NAFLD diagnosis for the HRI, HSI and NAFLD logit score were equal to 1.12, 56.1 and 0.59, and significantly differed from the cut-off values reported for the general population in the literature.
Conclusions: Our study confirms the usefulness of only two out of three techniques, the HRI and the NAFLD logit score for the diagnosis of NAFLD in patients with severe obesity. Methods designed for the general population require different cut-off values to achieve accurate performance in severe obesity.

biomarkers, fatty liver disease, hepatorenal index, obesity, ultrasound

In this paper, we propose a novel deep learning method for joint classification and segmentation of breast masses based on radio-frequency (RF) ultrasound (US) data. In comparison to commonly used classification and segmentation techniques, utilizing B-mode US images, we train the network with RF data (data before envelope detection and dynamic compression), which are considered to include more information on tissue’s physical properties than standard B-mode US images. Our multi-task network, based on the Y-Net architecture, can effectively process large matrices of RF data by mixing 1D and 2D convolutional filters. We use data collected from 273 breast masses to compare the performance of networks trained with RF data and US images. The multi-task model developed based on the RF data achieved good classification performance, with area under the receiver operating characteristic curve (AUC) of 0.90. The network based on the US images achieved AUC of 0.87. In the case of the segmentation, we obtained mean Dice scores of 0.64 and 0.60 for the approaches utilizing US images and RF data, respectively. Moreover, the interpretability of the networks was studied using class activation mapping technique and by filter weights visualizations.

breast mass classification, breast mass segmentation, convolutional neural networks, deep learning, quantitative ultrasound, ultrasound imagin

Temperature monitoring in ultrasound (US) imaging is important for various medical treatments, such as high-intensity focused US (HIFU) therapy or hyperthermia. In this work, we present a deep learning based approach to temperature monitoring based on radio-frequency (RF) US data. We used Siamese neural networks in an unsupervised way to spatially compare RF data collected at different time points of the heating process. The Siamese model consisted of two identical networks initially trained on a large set of simulated RF data to assess tissue backscattering properties. To illustrate our approach, we experimented with a tissue-mimicking phantom and an ex-vivo tissue sample, which were both heated with a HIFU transducer. During the experiments, we collected RF data with a regular US scanner. To determine spatiotemporal variations in temperature distribution within the samples, we extracted small 2D patches of RF data and compared them with the Siamese network. Our method achieved good performance in determining the spatiotemporal distribution of temperature during heating. Compared with the temperature monitoring based on the change in radio-frequency signal backscattered energy parameter, our method provided more smooth spatial parametric maps and did not generate ripple artifacts. The proposed approach, when fully developed, might be used for US based temperature.

temperature monitoring, high intensity ultrasound, deep learning, transfer learning, ultrasound imaging

Objective. Prediction of the response to neoadjuvant chemotherapy (NAC) in breast cancer is important for patient outcomes. In this work, we propose a deep learning based approach to NAC response prediction in ultrasound (US) imaging. Approach. We develop recurrent neural networks that can process serial US imaging data to predict chemotherapy outcomes. We present models that can process either raw radio-frequency (RF) US data or regular US images. The proposed approach is evaluated based on 204 sequences of US data from 51 breast cancers. Each sequence included US data collected before the chemotherapy and after each subsequent dose, up to the 4th course. We investigate three pre-trained convolutional neural networks (CNNs) as back-bone feature extractors for the recurrent network. The CNNs were pre-trained using raw US RF data, US b-mode images and RGB images from the ImageNet dataset. The first two networks were developed using US data collected from malignant and benign breast masses. Main results. For the pre-treatment data, the better performing network, with back-bone CNN pre-trained on US images, achieved area under the receiver operating curve (AUC) of 0.81 (±0.04). Performance of the recurrent networks improved with each course of the chemotherapy. For the 4th course, the better performing model, based on the CNN pre-trained with RGB images, achieved AUC value of 0.93 (±0.03). Statistical analysis based on the DeLong test presented that there were no significant differences in AUC values between the pre-trained networks at each stage of the chemotherapy (p-values > 0.05). Significance. Our study demonstrates the feasibility of using recurrent neural networks for the NAC response prediction in breast cancer US.

breast cancer, deep learning, neoadjuvant chemotherapy, reccurent neural networks, ultrasound imaging

Aim of the study: Deep neural networks have achieved good performance in breast mass classification in ultrasound imaging. However, their usage in clinical practice is still lim¬ited due to the lack of explainability of decisions conducted by the networks. In this study, to address the explainability problem, we generated saliency maps indicating ultrasound image regions important for the network’s classification decisions. Material and methods: Ultrasound images were collected from 272 breast masses, including 123 malignant and 149 benign. Transfer learning was applied to develop a deep network for breast mass clas¬sification. Next, the class activation mapping technique was used to generate saliency maps for each image. Breast mass images were divided into three regions: the breast mass region, the peritumoral region surrounding the breast mass, and the region below the breast mass. The pointing game metric was used to quantitatively assess the overlap between the saliency maps and the three selected US image regions. Results: Deep learning classifier achieved the area under the receiver operating characteristic curve, accuracy, sensitivity, and specific¬ity of 0.887, 0.835, 0.801, and 0.868, respectively. In the case of the correctly classified test US images, analysis of the saliency maps revealed that the decisions of the network could be associated with the three selected regions in 71% of cases. Conclusions: Our study is an important step toward better understanding of deep learning models developed for breast mass diagnosis. We demonstrated that the decisions made by the network can be related to the appearance of certain tissue regions in breast mass US images.

deep learning, breast mass diagnosis, attention maps, explainability

Ultrasound (US) imaging is widely used to help radiologists in diagnosing breast cancer. In this work, we propose a deep learning based approach to breast mass classification in US. Transfer learning with convolutional neural networks (CNNs) is commonly used to develop object recognition models in medical image analysis. The most widely used fine-tuning techniques aim to modify weights of pre-trained networks to address target medical problems. However, fine-tuning can be difficult when the number of trainable parameters of the pre-trained network is large and the available medical data are scarce. To address this issue, we propose a novel transfer learning technique based on deep representation scaling (DRS) layers, which are inserted between the blocks of a pre-trained CNN to enable better flow of information in the network. During network training, we only update the parameters of the DRS layers in order to adjust the pre-trained CNN to process breast mass US images. We present that the DRS based approach greatly reduces the number of trainable parameters, and achieves better or comparable performance to the standard transfer learning techniques. The proposed DRS layer method combined with the standard fine-tuning techniques achieved excellent breast mass classification performance, with area under the receiver operating characteristic curve of 0.955 and accuracy of 0.915.

breast mass classification, convolutional neural networks, deep learning, transfer learning, ultrasound imaging

Early prediction of response to neoadjuvant chemotherapy (NAC) in breast cancer is crucial for guiding therapy decisions. In this work, we propose a deep learning based approach for the early NAC response prediction in ultrasound (US) imaging. We used transfer learning with deep convolutional neural networks (CNNs) to develop the response prediction models. The usefulness of two transfer learning techniques was examined. First, a CNN pre-trained on the ImageNet dataset was utilized. Second, we applied double transfer learning, the CNN pre-trained on the ImageNet dataset was additionally fine-tuned with breast mass US images to differentiate malignant and benign lesions. Two prediction tasks were investigated. First, a L1 regularized logistic regression prediction model was developed based on generic neural features extracted from US images collected before the chemotherapy (a priori prediction). Second, Siamese CNNs were used to quantify differences between US images collected before the treatment and after the first and second course of NAC. The proposed methods were evaluated using US data collected from 39 tumors. The better performing deep learning models achieved areas under the receiver operating characteristic curve of 0.797 and 0.847 in the case of the a priori prediction and the Siamese model, respectively. The proposed approach was compared with a
method based on handcrafted morphological features. Our study presents the feasibility of using transfer learning with CNNs for the NAC response prediction in US imaging.

breast cancer, deep learning, neoadjuvant chemotherapy, Siamese convolutional neural networks, ultrasound imaging

Objective: To develop a fully automated full-thickness cartilage segmentation and mapping of T1, T1ρ, and T2*, as well as macromolecular fraction (MMF) by combining a series of quantitative 3D ultrashort echo time (UTE) cones MR imaging with a transfer learning–based U-Net convolutional neural networks (CNN) model. Methods: Sixty-five participants (20 normal, 29 doubtful-minimal osteoarthritis (OA), and 16 moderate-severe OA) were scanned using 3D UTE cones T1 (Cones-T1), adiabatic T1ρ (Cones-AdiabT1ρ), T2* (Cones-T2*), and magnetization transfer (Cones-MT) sequences at 3 T. Manual segmentation was performed by two experienced radiologists, and automatic segmentation was completed using the proposed U-Net CNN model. The accuracy of cartilage segmentation was evaluated using the Dice score and volumetric overlap error (VOE). Pearson correlation coefficient and intraclass correlation coefficient (ICC) were calculated to evaluate the consistency of quantitative MR parameters extracted from automatic and manual segmentations. UTE biomarkers were compared among different subject groups using one-way ANOVA. Results: The U-Net CNN model provided reliable cartilage segmentation with a mean Dice score of 0.82 and a mean VOE of 29.86%. The consistency of Cones-T1, Cones-AdiabT1ρ, Cones-T2*, and MMF calculated using automatic and manual segmentations ranged from 0.91 to 0.99 for Pearson correlation coefficients, and from 0.91 to 0.96 for ICCs, respectively. Significant increases in Cones-T1, Cones-AdiabT1ρ, and Cones-T2* (p < 0.05) and a decrease in MMF (p < 0.001) were observed in doubtful-minimal OA and/or moderate-severe OA over normal controls. Conclusion: Quantitative 3D UTE cones MR imaging combined with the proposed U-Net CNN model allows a fully automated comprehensive assessment of articular cartilage.

deep learning, cartilage, biomarkers, osteoarthritis

Background: Obesity increases and surgical weight reduction decreases the risk of atrial fibrillation (AF) and heart failure (HF). We hypothesized that surgically induced weight loss may favorably affect left atrial (LA) mechanical function measured by longitudinal strain, which has recently emerged as an independent imaging biomarker of increased AF and HF risk. Methods: We retrospectively evaluated echocardiograms performed before and 12.2 ± 2.2 months after bariatric surgery in 65 patients with severe obesity (mean age 39 [36; 47] years, 72% of females) with no known cardiac disease or arrhythmia. The LA mechanical function was measured by the longitudinal strain using the semi-automatic speckle tracking method. Results: After surgery, body mass index decreased from 43.72 ± 4.34 to 30.04 ± 4.33 kg/m2. We observed a significant improvement in all components of the LA strain. LA reservoir strain (LASR) and LA conduit strain (LASCD) significantly increased (35.7% vs 38.95%, p = 0.0005 and − 19.6% vs − 24.4%, p < 0.0001) and LA contraction strain (LASCT) significantly decreased (− 16% vs − 14%, p = 0.0075). There was a significant correlation between an increase in LASR and LASCD and the improvement in parameters of left ventricular diastolic and longitudinal systolic function (increase in E’ and MAPSE). Another significant correlation was identified between the decrease in LASCT and an improvement in LA function (decrease in A’). Conclusions: The left atrial mechanical function improves after bariatric surgery. It is partially explained by the beneficial effect of weight reduction on the left ventricular diastolic and longitudinal systolic function. This effect may contribute to decreased risk of AF and HF after bariatric surgery.

left atrial strain, bariatric surgery, atrial fibrillation, heart failure

Objectives - To develop and evaluate deep learning models devised for liver fat assessment based on ultrasound (US) images acquired from four different liver views: transverse plane (hepatic veins at the confluence with the inferior vena cava, right portal vein, right posterior portal vein) and sagittal plane (liver/kidney). Methods - US images (four separate views) were acquired from 135 participants with known or suspected nonalcoholic fatty liver disease. Proton density fat fraction (PDFF) values derived from chemical shift-encoded magnetic resonance imaging served as ground truth. Transfer learning with a deep convolutional neural network (CNN) was applied to develop models for diagnosis of fatty liver (PDFF ≥ 5%), diagnosis of advanced steatosis (PDFF ≥ 10%), and PDFF quantification for each liver view separately. In addition, an ensemble model based on all four liver view models was investigated. Diagnostic performance was assessed using the area under the receiver operating characteristics curve (AUC), and quantification was assessed using the Spearman correlation coefficient (SCC). Results - The most accurate single view was the right posterior portal vein, with an SCC of 0.78 for quantifying PDFF and AUC values of 0.90 (PDFF ≥ 5%) and 0.79 (PDFF ≥ 10%). The ensemble of models achieved an SCC of 0.81 and AUCs of 0.91 (PDFF ≥ 5%) and 0.86 (PDFF ≥ 10%). Conclusion - Deep learning-based analysis of US images from different liver views can help assess liver fat.

attention mechanism, convolutional neural networks, deep learning, nonalcoholic fatty liver disease, proton density fat fraction, ultrasound images

Background: Radiofrequency ultrasound data from the liver contain rich information about liver microstructure and composition. Deep learning might exploit such information to assess nonalcoholic fatty liver disease (NAFLD). Purpose: To develop and evaluate deep learning algorithms that use radiofrequency data for NAFLD assessment, with MRI-derived proton density fat fraction (PDFF) as the reference. Materials and Methods: A HIPAA-compliant secondary analysis of a single-center prospective study was performed for adult participants with NAFLD and control participants without liver disease. Participants in the parent study were recruited between February 2012 and March 2014 and underwent same-day US and MRI of the liver. Participants were randomly divided into an equal number of training and test groups. The training group was used to develop two algorithms via cross-validation: a classifier to diagnose NAFLD (MRI PDFF ≥ 5%) and a fat fraction estimator to predict MRI PDFF. Both algorithms used one-dimensional convolutional neural networks. The test group was used to evaluate the classifier for sensitivity, specificity, positive predictive value, negative predictive value, and accuracy and to evaluate the estimator for correlation, bias, limits of agreements, and linearity between predicted fat fraction and MRI PDFF. Results: A total of 204 participants were analyzed, 140 had NAFLD (mean age, 52 years ± 14 [standard deviation]; 82 women) and 64 were control participants (mean age, 46 years ± 21; 42 women). In the test group, the classifier provided 96% (95% confidence interval [CI]: 90%, 99%) (98 of 102) accuracy for NAFLD diagnosis (sensitivity, 97% [95% CI: 90%, 100%], 68 of 70; specificity, 94% [95% CI: 79%, 99%], 30 of 32; positive predictive value, 97% [95% CI: 90%, 99%], 68 of 70; negative predictive value, 94% [95% CI: 79%, 98%], 30 of 32). The estimator-predicted fat fraction correlated with MRI PDFF (Pearson r = 0.85). The mean bias was 0.8% (P = .08), and 95% limits of agreement were -7.6% to 9.1%. The predicted fat fraction was linear with an MRI PDFF of 18% or less (r = 0.89, slope = 1.1, intercept = 1.3) and nonlinear with an MRI PDFF greater than 18%. Conclusion: Deep learning algorithms using radiofrequency ultrasound data are accurate for diagnosis of nonalcoholic fatty liver disease and hepatic fat fraction quantification when other causes of steatosis are excluded.

In this work, we propose a deep learning method for breast mass segmentation in ultrasound (US). Variations in breast mass size and image characteristics make the automatic segmentation difficult. To addressthis issue, we developed a selective kernel (SK) U-Net convolutional neural network. The aim of the SKswas to adjust network's receptive fields via an attention mechanism, and fuse feature maps extractedwith dilated and conventional convolutions. The proposed method was developed and evaluated usingUS images collected from 882 breast masses. Moreover, we used three datasets of US images collectedat different medical centers for testing (893 US images). On our test set of 150 US images, the SK-U-Netachieved mean Dice score of 0.826, and outperformed regular U-Net, Dice score of 0.778. When evaluatedon three separate datasets, the proposed method yielded mean Dice scores ranging from 0.646 to 0.780. Additional fine-tuning of our better-performing model with data collected at different centers improvedmean Dice scores by ~6%. SK-U-Net utilized both dilated and regular convolutions to process US images. We found strong correlation, Spearman's rank coefficient of 0.7, between the utilization of dilated convo-lutions and breast mass size in the case of network's expansion path. Our study shows the usefulness ofdeep learning methods for breast mass segmentation. SK-U-Net implementation and pre-trained weightscan be found at github.com/mbyr/bus_seg.

attention mechanism, breast mass segmentation, convolutional neural networks, deep learning, receptive field, ultrasound imaging

Purpose: To develop a deep learning-based method for knee menisci segmentation in 3D ultrashort echo time (UTE) cones MR imaging, and to automatically determine MR relaxation times, namely the T1, T1ρ, and T2* parameters, which can be used to assess knee osteoarthritis (OA). Methods: Whole knee joint imaging was performed using 3D UTE cones sequences to collect data from 61 human subjects. Regions of interest (ROIs) were outlined by 2 experienced radiologists based on subtracted T1ρ-weighted MR images. Transfer learning was applied to develop 2D attention U-Net convolutional neural networks for the menisci segmentation based on each radiologist's ROIs separately. Dice scores were calculated to assess segmentation performance. Next, the T1, T1ρ, T2* relaxations, and ROI areas were determined for the manual and automatic segmentations, then compared. Results: The models developed using ROIs provided by 2 radiologists achieved high Dice scores of 0.860 and 0.833, while the radiologists' manual segmentations achieved a Dice score of 0.820. Linear correlation coefficients for the T1, T1ρ, and T2* relaxations calculated using the automatic and manual segmentations ranged between 0.90 and 0.97, and there were no associated differences between the estimated average meniscal relaxation parameters. The deep learning models achieved segmentation performance equivalent to the inter-observer variability of 2 radiologists. Conclusion: The proposed deep learning-based approach can be used to efficiently generate automatic segmentations and determine meniscal relaxations times. The method has the potential to help radiologists with the assessment of meniscal diseases, such as OA.

deep learning, menisci, osteoarthritis, quantitative MR, segmentation

We propose a novel approach to breast mass classification based on deep learning models that utilize raw radio-frequency (RF) ultrasound (US) signals. US images, typically displayed by US scanners and used to develop computer-aided diagnosis systems, are reconstructed using raw RF data. However, information related to physical properties of tissues present in RF signals is partially lost due to the irreversible compression necessary to make raw data readable to the human eye. To utilize the information present in raw US data, we develop deep learning models that can automatically process small 2D patches of RF signals and their amplitude samples. We compare our approach with classification method based on the Nakagami parameter, a widely used quantitative US technique utilizing RF data amplitude samples. Our better performing deep learning model, trained using RF signals and their envelope samples, achieved good classification performance, with the area under the receiver attaining operating characteristic curve (AUC) and balanced accuracy of 0.772 and 0.710, respectively. The proposed method significantly outperformed the Nakagami parameter-based classifier, which achieved AUC and accuracy of 0.64 and 0.611, respectively. The developed deep learning models were used to generate parametric maps illustrating the level of mass malignancy. Our study presents the feasibility of using RF data for the development of deep learning breast mass classification models.

breast lesion classification, convolutional neural networks, deep learning, radio-frequency signals, ultrasound imaging

Objectives: To assess the feasibility of using ultrasound (US) image features related to the median nerve echogenicity and shape for carpal tunnel syndrome (CTS) diagnosis. Methods: In 31 participants (21 healthy participants and 10 patients with CTS), US images were collected with a 30-MHz transducer from median nerves at the wrist crease in 2 configurations: a neutral position and with wrist extension. Various morphologic features, including the cross-sectional area (CSA), were calculated to assess the nerve shape. Carpal tunnel syndrome commonly results in loss of visualization of the nerve fascicular pattern on US images. To assess this phenomenon, we developed a nerve-tissue contrast index (NTI) method. The NTI is a ratio of average brightness levels of surrounding tissue and the median nerve, both calculated on the basis of a US image. The area under the curve (AUC) from a receiver operating characteristic curve analysis and t test were used to assess the usefulness of the features for differentiation of patients with CTS from control participants. Results: We obtained significant differences in the CSA and NTI parameters between the patients with CTS and control participants (P < .01), with the corresponding highest AUC values equal to 0.885 and 0.938, respectively. For the remaining investigated morphologic features, the AUC values were less than 0.685, and the differences in means between the patients and control participants were not statistically significant (P > .10). The wrist configuration had no impact on differences in average parameter values (P > .09). Conclusions: Patients with CTS can be differentiated from healthy individuals on the basis of the median nerve CSA and echogenicity. Carpal tunnel syndrome is not manifested in a change of the median nerve shape that could be related to circularity or contour variability.

carpal tunnel syndrome, cross-sectional area, echogenicity, median nerve, morphologic features, ultrasound

Purpose: We propose a deep learning-based approach to breast mass classification in sonographyand compare it with the assessment of four experienced radiologists employing breast imagingreporting and data system 4th edition lexicon and assessment protocol. Methods: Several transfer learning techniques are employed to develop classifiers based on a set of882 ultrasound images of breast masses. Additionally, we introduce the concept of a matching layer. The aim of this layer is to rescale pixel intensities of the grayscale ultrasound images and convertthose images to red, green, blue (RGB) to more efficiently utilize the discriminative power of theconvolutional neural network pretrained on the ImageNet dataset. We present how this conversioncan be determined during fine-tuning using back-propagation. Next, we compare the performance ofthe transfer learning techniques with and without the color conversion. To show the usefulness of ourapproach, we additionally evaluate it using two publicly available datasets. Results: Color conversion increased the areas under the receiver operating curve for each transferlearning method. For the better-performing approach utilizing the fine-tuning and the matching layer,the area under the curve was equal to 0.936 on a test set of 150 cases. The areas under the curves forthe radiologists reading the same set of cases ranged from 0.806 to 0.882. In the case of the two sepa-rate datasets, utilizing the proposed approach we achieved areas under the curve of around 0.890. Conclusions: The concept of the matching layer is generalizable and can be used to improve theoverall performance of the transfer learning techniques using deep convolutional neural networks. When fully developed as a clinical tool, the methods proposed in this paper have the potential to helpradiologists with breast mass classification in ultrasound.

BI-RADS, breast mass classification, convolutional neural networks, transfer learning, ultrasound imaging

Purpose: Quantitative imaging methods could improve diagnosis of rotator cuff degeneration, but the capability of quantitative MR and US imaging parameters to detect alterations in collagen is unknown. The goal of this study was to assess quantitative MR and US imaging measures for detecting abnormalities in collagen using an in vitro model of tendinosis with biochemical and histological correlation. Method: 36 pieces of supraspinatus tendons from 6 cadaveric donors were equally distributed into 3 groups (2 subjected to different concentrations of collagenase and a control group). Ultrashort echo time MR and US imaging measures were performed to assess changes at baseline and after 24 h of enzymatic digestion. Biochemical and histological measures, including brightfield, fluorescence, and polarized microscopy, were used to verify the validity of the model and were compared with quantitative imaging parameters. Correlations between the imaging parameters and biochemically measured digestion were analyzed. Results: Among the imaging parameters, macromolecular fraction (MMF), adiabatic T1p, T2*, and backscatter coefficient (BSC) were useful in differentiating between the extent of degeneration among the 3 groups. MMF strongly correlated with collagen loss (r=-0.81; 95% confidence interval [CI]: -0.90,-0.66), while the adiabatic T1p (r = 0.66; CI: 0.42,0.81), T2* (r = 0.58; CI: 0.31,0.76), and BSC (r = 0.51; CI: 0.22,0.72) moderately correlated with collagen loss. Conclusions: MMF, adiabatic T1p, and T2* measured and US BSC can detect alterations in collagen. Of the quantitative MR and US imaging measures evaluated, MMF showed the highest correlation with collagen loss and can be used to assess rotator cuff degeneration.

rotator cuff tendon, tendinopathy, quantitative MRI, UTE, quantitative ultrasound

We investigate the usefulness of quantitative ultrasound and B-mode texture features for characterization of ulnar nerve fascicles. Ultrasound data were acquired from cadaveric specimens using a nominal 30-MHz probe. Next, the nerves were extracted to prepare histology sections. Eighty-five fascicles were matched between the B-mode images and the histology sections. For each fascicle image, we selected an intra-fascicular region of interest. We used histology sections to determine features related to the concentration of collagen and myelin and ultrasound data to calculate the backscatter coefficient (–24.89 ± 8.31 dB), attenuation coefficient (0.92 ± 0.04 db/cm-MHz), Nakagami parameter (1.01 ± 0.18) and entropy (6.92 ± 0.83), as well as B-mode texture features obtained via the gray-level co-occurrence matrix algorithm. Significant Spearman rank correlations between the combined collagen and myelin concentrations were obtained for the backscatter coefficient (R = –0.68), entropy (R = –0.51) and several texture features. Our study indicates that quantitative ultrasound may potentially provide information on structural components of nerve fascicles.

nerve, quantitative ultrasound, high frequency, histology, pattern recognition, texture analysis

Purpose
The nonalcoholic fatty liver disease is the most common liver abnormality. Up to date, liver biopsy is the reference standard for direct liver steatosis quantification in hepatic tissue samples. In this paper we propose a neural network-based approach for nonalcoholic fatty liver disease assessment in ultrasound.
Methods
We used the Inception-ResNet-v2 deep convolutional neural network pre-trained on the ImageNet dataset to extract high-level features in liver B-mode ultrasound image sequences. The steatosis level of each liver was graded by wedge biopsy. The proposed approach was compared with the hepatorenal index technique and the gray-level co-occurrence matrix algorithm. After the feature extraction, we applied the support vector machine algorithm to classify images containing fatty liver. Based on liver biopsy, the fatty liver was defined to have more than 5% of hepatocytes with steatosis. Next, we used the features and the Lasso regression method to assess the steatosis level.
Results
The area under the receiver operating characteristics curve obtained using the proposed approach was equal to 0.977, being higher than the one obtained with the hepatorenal index method, 0.959, and much higher than in the case of the gray-level co-occurrence matrix algorithm, 0.893. For regression the Spearman correlation coefficients between the steatosis level and the proposed approach, the hepatorenal index and the gray-level co-occurrence matrix algorithm were equal to 0.78, 0.80 and 0.39, respectively.
Conclusions
The proposed approach may help the sonographers automatically diagnose the amount of fat in the liver. The presented approach is efficient and in comparison with other methods does not require the sonographers to select the region of interest.

Nonalcoholic fatty, liver disease, Ultrasound imaging Deep learning, Convolutional neural networks, Hepatorenal index, Transfer learning

Quantitative ultrasound has been widely used for tissue characterization. In this paper we propose a new approach for tissue compression assessment. The proposed method employs the relation between the tissue scatterers' local spatial distribution and the resulting frequency power spectrum of the backscat- tered ultrasonic signal. We show that due to spatial distribution of the scatterers, the power spectrum exhibits characteristic variations. These variations can be extracted using the empirical mode decomposition and analyzed. Validation of our approach is performed by simulations and in-vitro experiments using a tissue sample under compression. The scatterers in the compressed tissue sample approach each other and consequently, the power spectrum of the backscattered signal is modified. We present how to assess this phenomenon with our method. The proposed in this paper approach is general and may provide useful information on tissue scattering properties.

tissue characterization, tissue compression, quantitative ultrasound, empirical mode decomposition, signal anaysis

Ultrasound imaging is widely used for breast lesion differentiation. In this paper we propose a neural transfer learning method for breast lesion classification in ultrasound. As reported in several papers, the content and the style of a particular image can be separated with a convolutional neural network. The style, coded by the Gram matrix, can be used to perform neural transfer of artistic style. In this paper we extract the neural style representations of malignant and benign breast lesions using the VGG19 neural network. Next, the Fisher discriminant analysis is used to separate those neural style representations and perform classification. The proposed approach achieves good classification performance (AUC of 0.847). Our method is compared with another transfer learning technique based on extracting pooling layer features (AUC of 0.826). Moreover, we apply the Fisher discriminant analysis to differentiate breast lesions using ultrasound images (AUC of 0.758). Additionally, we extract the eigenimages related to malignant and benign breast lesions and show that these eigenimages present features commonly associated with lesion type, such as contour attributes or shadowing. The proposed techniques may be useful for the researchers interested in ultrasound breast lesion characterization.

Breast lesions classification, Deep learning, Discriminant analysis, Transfer learning, Ultrasound imaging

Purpose: The aim of this paper is to provide access to a database consisting of the raw radio-frequency ultrasonic echoes acquired from malignant and benign breast lesions. The database is freely available for study and signal analysis. Acquisition and validation methods: The ultrasonic radio-frequency echoes were recorded from breast focal lesions of patients of the Institute of Oncology in Warsaw. The data were collected between 11/2013 and 10/2015. Patients were examined by a radiologist with 18 yr' experience in the ultrasonic examination of breast lesions. The set of data includes scans from 52 malignant and 48 benign breast lesions recorded in a group of 78 women. For each lesion, two individual orthogonal scans from the pathological region were acquired with the Ultrasonix SonixTouch Research ultrasound scanner using the L14-5/38 linear array transducer. All malignant lesions were histologically assessed by core needle biopsy. In the case of benign lesions, part of them was histologically assessed and another part was observed over a 2-year period. Data format and usage notes: The radio-frequency echoes were stored in Matlab file format. For each scan, the region of interest was provided to correctly indicate the lesion area. Moreover, for each lesion, the BI-RADS category and the lesion class were included. Two code examples of data manipulation are presented. The data can be downloaded via the Zenodo repository (https://doi.org/10.5281/zenodo.545928) or the website http ://bluebox.ippt.gov.pl/~hpiotrzk. Potential applications: The database can be used to test quantitative ultrasound techniques and ultrasound image processing algorithms, or to develop computer-aided diagno sis systems.

breast lesions, dataset, ultrasonic signals, ultrasonography

A technique using pulsed High Intensity Focused Ultrasound (HIFU) to destroy deep-seated solid tumors is a promising noninvasive therapeutic approach. A main purpose of this study was to design and test a HIFU transducer suitable for preclinical studies of efficacy of tested, anti-cancer drugs, activated by HIFU beams, in the treatment of a variety of solid tumors implanted to various organs of small animals at the depth of the order of 1–2 cm under the skin. To allow focusing of the beam, generated by such transducer, within treated tissue at different depths, a spherical, 2-MHz, 29-mm diameter annular phased array transducer was designed and built. To prove its potential for preclinical studies on small animals, multiple thermal lesions were induced in a pork loin ex vivo by heating beams of the same: 6 W, or 12 W, or 18 W acoustic power and 25 mm, 30 mm, and 35 mm focal lengths. Time delay for each annulus was controlled electronically to provide beam focusing within tissue at the depths of 10 mm, 15 mm, and 20 mm. The exposure time required to induce local necrosis was determined at different depths using thermocouples. Location and extent of thermal lesions determined from numerical simulations were compared with those measured using ultrasound and magnetic resonance imaging techniques and verified by a digital caliper after cutting the tested tissue samples. Quantitative analysis of the results showed that the location and extent of necrotic lesions on the magnetic resonance images are consistent with those predicted numerically and measured by caliper. The edges of lesions were clearly outlined although on ultrasound images they were fuzzy. This allows to conclude that the use of the transducer designed offers an effective noninvasive tool not only to induce local necrotic lesions within treated tissue without damaging the surrounding tissue structures but also to test various chemotherapeutics activated by the HIFU beams in preclinical studies on small animals.

spherical annular phased array transducer, pulsed HIFU beam, electronically adjustable focal length, local tissue heating, thermal ablation, necrotic lesion

Temperature monitoring is essential for various medical treatments. In this work, we investigate the impact of temperature on backscattered ultrasound echo statistics during a high intensity focused ultrasound treatment. A tissue mimicking phantom was heated with a spherical ultrasonic transducer up to 56 _C in order to imitate tissue necrosis. During the heating, an imaging scanner was used to acquire backscattered echoes from the heated region. These data was then modeled with the homodyned K distribution. We found that the best temperature indicator can be obtained by combining two parameters of the model, namely the backscattered echo mean intensity and the effective number of scatterers per resolution cell. Next, ultrasonic thermometer was designed and used to create a map of the temperature induced within the tissue phantom during the treatment

Temperature monitoring, homodyned K distribution, focused ultrasound

Purpose:
Statistical modeling of an ultrasound backscattered echo envelope is used for tissue characterization. However, in the presence of complex structures within the analyzed area, estimation of parameters is disturbed and unreliable, e.g., in the case of breast tumor classification. In order to improve the differentiation of breast lesions, the authors proposed a method based on the segmentation of homodyned K distribution parameter maps. Regions within lesions of different scattering properties were extracted and analyzed. In order to improve the classification, the best-performing features were selected from various regions and then combined.

Methods:
A radio-frequency data set consisting of 103 breast lesions was used in the authors’ analysis. Maps of homodyned K distribution parameters were created using an algorithm based on signal-to-noise ratio, kurtosis, and skewness of fractional-order envelope moments. A Markov random field model was used to segment parametric maps. Features of different segments were extracted and evaluated based on bootstrapping and the receiver operating characteristic curve. To determine the best-performing feature subset, the authors applied the joint mutual information criterion.

Results:
It was found that there were individual features which performed better than the ones commonly used for lesion characterization, like the parameter obtained through averaging of values over the whole lesion. The authors selected and discussed the best-performing features. Properties of different extracted regions were important and improved the distinction between benign and malignant tumors. The best performance was obtained by combining four features with the area under the receiver operating curve of 0.84.

Conclusions:
The study showed that the analysis of internal changes in lesion parametric maps leads to a better classification of breast tumors. The authors recommend combining multiple features for characterization, instead of using only one parameter, especially in the case of heterogeneous lesions.

Cancer, Ultrasonography, Backscattering, Data sets, Medical image noise

Texture of ultrasound images contain information about the properties of examined tissues. The analysis of statistical properties of backscattered ultrasonic echoes has been recently successfully applied to differentiate healthy breast tissue from the benign and malignant lesions. We propose a novel procedure of tissue characterization based on acquiring backscattered echoes from the heated breast. We have proved that the temperature increase inside the breast modifies the intensity, spectrum of the backscattered signals and the probability density function of envelope samples. We discuss the differences in probability density functions in two types of tissue regions, e.g. cysts and the surrounding glandular tissue regions. Independently, Pennes bioheat equation in heterogeneous breast tissue was used to describe the heating process. We applied the finite element method to solve this equation. Results have been compared with the ultrasonic predictions of the temperature distribution. The results confirm the possibility of distinguishing the differences in thermal and acoustical properties of breast cyst and surrounding glandular tissues.

medical ultrasound, temperature changes in vivo, breast tissue, ultrasonic temperature measurement

Polyvinyl-alcohol cryogel is commonly used for soft tissue phantom manufacture. The gel formation from an aqueous solution of polyvinyl-alcohol takes place during the freezing and thawing cycle. The aim of this work was to assess the degree of gel solidification, hence the material stiffness, by means of quantitative ultrasound. We manufactured three phantoms which differed in the number of freezing/thawing cycles. First, tissue phantoms were examined with an elastography technique. Next, we measured the speed of sound and the attenuation coefficient. What is more, the inter structure variations in phantoms were assessed with the Nakagami imaging which quantifies the scattering properties of the backscattered ultrasound echo. Obtained results confirmed the connection between the number of freezing/thawing cycles and the solidification process. We defined the boundary layer as a region which has a different structure than the sample interior. Next, for each phantom this layer was extracted based on a Nakagami parameter map. We calculated that the thickness of the boundary layer was lower in samples which were subjected to a larger number of freezing/thawing cycles.

soft tissue phantoms, elastography, ultrasound attenuation, speed of sound, Nakagami maps, stiffness

Quantitative ultrasound methods are widely investigated as a promising tool for tissue characterization. In this paper, a novel quantitative method is developed which can be used to assess scattering properties of tissues. The proposed method is based on analysis of oscillations of the backscattered echo power spectrum. It is shown that these oscillations of the power spectrum are connected with the distances between scatterers within the medium. Two techniques are proposed to assess the scatterer’s distribution. First, we show that the inverse Fourier transform of the backscattered echo power spectrum corresponds to a histogram of the distances between scatterers. Second, the Hilbert-Huang transform is used to directly extract the power spectrum oscillations. Both methods are examined by means of a numerical experiment. A cellular gas model of a biological medium is considered. Results are presented and discussed. Both methods can be used to evaluate the scatterer’s distribution by means of the power spectrum oscillations.

quantitative ultrasound, signal analysis, wave scattering

Different ultrasound echoes properties have been used for the noninvasive temperature monitoring. Temperature variations that occur during heating/cooling process induce changes in a random process of ultrasound backscattering. It was already proved that the probability distribution of the backscattered RF (radio frequency) signals is sensitive to the temperature variations. Contrary to previously used methods which explored models of scattering and involved techniques of fitting histograms to a special probability distribution two more direct measures of changes in statistics are proposed in this paper as temperature markers. They measure the ”distance” between the probability distributions. The markers are the Kolmogorov Smirnov distance and Kulback-Leiber divergence. The feasibility of using such nonparametric statistics for noninvasive ultrasound temperature estimation is demonstrated on the ultrasounds data collected during series of heating experiments in which the temperature was independently registered by the classical thermometer or thermocouples.

ultrasoud echoes, non-invasive temperaturę monitoring, Kolmogorov Smirnov distance, Kulback-Leiber divergence

Nowadays there are new modalities in ultrasound imaging allowing better characterization of tissue regions with different stiffness. We are proposing a novel approach based on compression and rarefaction of tissue simultaneously with imaging. The propagating wave is a combination of two pulses. A low frequency pulse is expected to change the local scattering properties of the tissue due to compression/rarefaction while a high frequency pulse is used for imaging. Two transmissions are performed for each scanning line. First, with the imaging pulse that propagates on maximum compression caused by a low frequency wave. Next, the low frequency wave is inverted and the imaging pulse propagates over the maximum rarefaction. After the processing of the subtracted echoes from subsequent transmissions including wavelet transform and band-pass filtering, differential images were reconstructed. The low frequency wave has a visible impact on the scattering properties of the tissue which can be observed on a differential image.

Segmentation of biomedical images is often ambiguous and complicated by noise, varying
contrasts, and imaging artifacts. We address the challenge of segmenting images of brain
tissue in which gene expression has been localized using in situ hybridization. Since gene
expression patterns differ widely between genes, it can be difficult to correctly discriminate
pixels positive for gene expression. In testing different segmentation networks, we observed
that each network had its own trade-offs between sensitivity and precision. To exploit
the benefits of all trained networks, we developed a meta-network that learns to combine
multiple segmentation maps from diverse segmentation architectures to generate a final
segmentation that best matches the ground-truth label. In our experiments, the meta-
network outperforms ensembles that simply average segmentation maps.

meta-learning, segmentation, gene expression

Background, Motivation and Objective
Flexible ultrasound (US) arrays are a promising technology that may further democratize US technology — e.g. in wearable US. Flexible transducers also pose challenges in image reconstruction, as they require adaptable beamforming delays due to a changing geometry of the probe. Various approaches have been proposed for flexible array shape estimation and beamforming, e.g. external sensors, deep learning and optimization. In this work, we propose a deep reinforcement learning (DRL) approach, where a software agent is responsible for tracking the array shape to properly reconstruct US B-mode image.

Statement of Contribution/Methods
Here we considered a reinforcement learning environment as a setup consisting of a US system with the flexible array and point targets phantom. The environment was simulated using j-Wave software. The environment’s state consisted of the current shape of the array, modeled as a sinusoid s = a sin(bx + c), and the current model of the array assumed by the beamformer: s’ = a’ sin(b’x + c’) (single-element STA scheme). A single episode consisted of
7 steps; the parameters a, b, c could vary from step to step (within bounds of physical constraints). The agent observed the current B-mode, could modify the current values of a’, b’ and c’ (action) and received a reward equal to the linear combination of the coherence factor and structural similarity index measure (SSIM) between the current and reference image. We trained our agent using TD3 approach and tested it for various settings of a, b and c.

Results/Discussion
Our agent achieved an average SSIM of 0.73 per episode step. Figure 1 shows the sequence of states and images within an example episode; the agent was able to correctly react to the change of the array shape. The DRL approach has the following advantages compared to other methods: the agent can be trained to operate in an environment with a changing state; and the agent can be trained to maximize expected return (dependent on beamforming quality metric), which does not have to be differentiable.

Fig. 1. (Top) B-mode images, estimated array shape (red) and actual shape (black) within an example episode. (Bottom) Coherence factor and SSIM achieved by the agent during the episode

Accurate estimation of the speed-of-sound (SoS) is important for ultrasound (US) image reconstruction techniques and tissue characterization. Various approaches have been proposed to calculate SoS, ranging from tomography-inspired algorithms like CUTE to convolutional networks, and more recently, physics-informed optimization frameworks based on differentiable beamforming. In this work, we utilize implicit neural representations (INRs) for SoS estimation in US. INRs are a type of neural network architecture that encodes continuous functions, such as images or physical quantities, through the weights of a network. Implicit networks may overcome the current limitations of SoS estimation techniques, which mainly arise from the use of non-adaptable and oversimplified physical models of tissue. Moreover, convolutional networks for SoS estimation, usually trained using simulated data, often fail when applied to real tissues due to out-of-distribution and data-shift issues. In contrast, implicit networks do not require extensive training datasets since each implicit network is optimized for an individual data case. This adaptability makes them suitable for processing US data collected from varied tissues and across different imaging protocols. We evaluated the proposed SoS estimation method based on INRs using data collected from a tissue-mimicking phantom containing four cylindrical inclusions, with SoS values ranging from 1480 m/s to 1600 m/s. The inclusions were immersed in a material with an SoS value of 1540 m/s. In experiments, the proposed method achieved strong performance, clearly demonstrating the usefulness of implicit networks for quantitative US applications.

beamforming, deep learning, implicit neural representations, speed-of-sound, quantitative ultrasound

We present the first automated pipeline to create an atlas of in situ hybridization gene expression in the adult marmoset brain in the same stereotaxic space. The pipeline consists of segmentation of gene expression from microscopy images and registration of images to a standard space. Automation of this pipeline is necessary to analyze the large volume of data in the genome-wide whole-brain dataset, and to process images that have varying intensity profiles and expression patterns with minimal human bias. To reduce the number of labelled images required for training, we develop a semi-supervised segmentation model. We further develop an iterative algorithm to register images to a standard space, enabling comparative analysis between genes and concurrent visualization with other datasets, thereby facilitating a more holistic understanding of primate brain structure and function.

contrastive learning, gene atlas, segmen-tation, semi-supervised learning, registration

We propose a novel image registration method based on implicit neural representations that addresses the challenging problem of registering a pair of brain images with similar anatomical structures, but where one image contains additional features or artifacts that are not present in the other image. To demonstrate its effectiveness, we use 2D microscopy in situ hybridization gene expression images of the marmoset brain. Accurately quantifying gene expression requires image registration to a brain template, which is difficult due to the diversity of patterns causing variations in visible anatomical brain structures. Our approach uses implicit networks in combination with an image exclusion loss to jointly perform the registration and decompose the image into a support and residual image. The support image aligns well with the template, while the residual image captures individual image characteristics that diverge from the template. In experiments, our method provided excellent results and outperformed other registration techniques.

brain, deep learning, gene expression, implicit neural representations, registration

We investigate the usefulness of formula-driven supervised learning (FDSL) for breast ultrasound (US) image analysis. Medical data are usually too scarce to develop a better performing deep learning model from scratch. Transfer learning with networks pre-trained on ImageNet is commonly applied to address this problem. FDSL techniques have been recently investigated as an alternative solution to ImageNet based approaches. In the FDSL setting, networks for transfer learning applications are developed using large amounts of synthetic images generated with mathematical formulas, possibly taking into account the characteristics of the target data. In this work, we use Field II to develop a large synthetic dataset of 100 000 US images presenting different contour objects, as shape features play an important role in breast mass characterization in US. Synthetic data are utilized to pre-train the ResNet50 classification model and various variants of the U-Net segmentation network. Next, the pre-trained models are fine-tuned on breast mass US images. Our results demonstrate that the proposed FDSL approach can provide good performance with respect to breast mass classification and segmentation.

breast cancer, deep learning, synthetic data, ultrasound

Standard classification methods based on hand-crafted morphological and texture features have achieved good performance in breast mass differentiation in ultrasound (US).
In comparison to deep neural networks, commonly perceived as ‘black-box’ models, classical techniques are based on features that have well-understood medical and physical interpretation. However, classifiers based on morphological features commonly
underperform in the presence of the shadowing artifact and ill-defined mass borders, while texture based classifiers may fail when the US image is too noisy. Therefore, in practice it would be beneficial to select the classification method based on the appearance of the particular US image. In this work, we develop a deep meta-network that can automatically process input breast mass US images and recommend whether to apply the shape or
texture based classifier for the breast mass differentiation. Our preliminary results demonstrate that meta-learning techniques can be used to improve the performance of the standard classifiers based on handcrafted features. With the proposed meta-learning based approach, we achieved the area under the receiver operating characteristic curve of 0.95 and accuracy of 0.91.

breast mass classification, deep learning, meta-learning, morphological features, texture features

Ultrasound (US) imaging is widely used for the tissue characterization. However, US images commonly suffer from speckle noise, which degrades perceived image quality. Various deep learning approaches have been proposed for US image denoising, but most of them lack the interpretability of how the network is processing the US image (black box problem). In this work, we utilize a deep reinforcement learning (RL) approach, the pixelRL, to US image denoising. The technique utilizes a set of easily interpretable and commonly used filtering operations applied in a pixel-wise manner. In RL, software agents act in an unknown environment and receive appropriate numerical rewards. In our case, each pixel of the input US image has an agent and state of the environment is the current US image. Agents iteratively denoise the US image by executing the following pixel-wise pre-defined actions: Gaussian, bilateral, median and box filtering, pixel value increment/decrement and no action. The proposed approach can be used to generate action maps depicting operations applied to process different parts of the US image. Agents were pre-trained on natural gray-scale images and evaluated on the breast mass US images. To enable the evaluation, we artificially corrupted the US images with noise. Compared with the reference (noise free US images), filtration of the images with the proposed method increased the average peak signal-to-noise ratio (PSNR) score from 14 dB to 26 dB and increased the structure similarity index score from 0.22 to 0.54. Our work confirms that it is feasible to use pixel-wise RL techniques for the US image denoising.

deep reinforcement learning, ultrasound imaging, image denoising, filtration, breast cancer

Convolutional neural networks (CNNs) have achieved remarkable success in medical image analysis tasks. In ultrasound (US) imaging, CNNs have been applied to object classification, image reconstruction and tissue characterization. However, CNNs can be vulnerable to adversarial attacks, even small perturbations applied to input data may significantly affect model performance and result in wrong output. In this work, we devise a novel adversarial attack, specific to ultrasound (US) imaging. US images are reconstructed based on radio-frequency signals. Since the appearance of US images depends on the applied image reconstruction method, we explore the possibility of fooling deep learning model by perturbing US B-mode image reconstruction method. We apply zeroth order optimization to find small perturbations of image reconstruction parameters, related to attenuation compensation and amplitude compression, which can result in wrong output. We illustrate our approach using a deep learning model developed for fatty liver disease diagnosis, where the proposed adversarial attack achieved success rate of 48%.

adversarial attacks, deep learning, fatty liver, transfer learning, ultrasound imaging

In this work we investigate the usefulness and robustness of transfer learning with deep convolutional neural networks (CNNs) for breast lesion classification in ultrasound (US). Deep learning models can be vulnerable to adversarial examples, engineered input image pixel intensities perturbations that force models to make classification errors. In US imaging, distribution of US image pixel intensities relies on applied US image reconstruction algorithm. We explore the possibility of fooling deep learning models for breast mass classification by modifying US image reconstruction method. Raw radio-frequency US signals acquired from malignant and benign breast masses were used to reconstruct US images, and develop classifiers using transfer learning with the VGG19, InceptionV3 and InceptionResNetV2 CNNs. The areas under the receiver operating characteristic curve (AUCs) obtained for each deep learning model developed and evaluated using US images reconstructed in the same way were equal to approximately 0.85, and there were no associated differences in AUC values between the models (DeLong test p-values > 0.15). However, due to small modifications of the US image reconstruction method the AUC values for the models utilizing the VGG19, InceptionV3 and InceptionResNetV2 CNNs significantly decreased to 0.592, 0.584 and 0.687, respectively. Our study shows that the modification of US image reconstruction algorithm can have significant negative impact on classification performance of deep models. Taking into account medical image reconstruction algorithms may help develop more robust deep learning computer aided diagnosis systems.

Adversarial attacks, Breast lesion classification, Computer aided diagnosis, Deep learning, Robustness, Ultrasound imaging, Transfer learning

In this paper we propose a computer-aided diagnosis system for the breast lesion classification. Our approach is based on quantitative ultrasound and deep learning. We used the Nakagami imaging to create parametric maps of breast lesions that illustrate tissue scattering properties. For this task the sliding window technique was applied. The Nakagami parameter was calculated using the maximum likelihood estimator. Next, we used the Nakagami parameter maps to train a convolutional neural network. Classification performance was evaluated by 5-fold cross-validation. We obtained the area under the receiver operating characteristic curve equal to 0.91. The results showed that our approach is useful to distinguishing between malignant and benign breast lesions. The proposed method serves as a general approach for tissue characterization and differentiation. The Nakagami parameter used in this study can be replaced with other QUS parameters and the neural network can be trained in a similar fashion.

Nakagami imaging, quantitative ultrasound, convolutional neural networks, breast lesion classification, deep learning

In this paper we investigate how to assess the ultrasound nonlinearity coefficient using plane wave imaging. We employ the technique based on excitation of the medium with ultrasonic pulses of increasing amplitude level. As the pulse pressure is increased, due to medium nonlinearity, higher fraction of energy is transferred from the fundamental to higher harmonics during the propagation. In this case the amplitude of the backscattered echo is not linear in respect to the initial pulse amplitude at source. This phenomenon can be used for the nonlinearity coefficient assessment and show its implementation for the plane wave imaging. The method was validated experimentally using a wire phantom immersed in water and scanned using the Verasonics scanner. We discuss the usefulness of the proposed technique and its shortcomings. In comparison to other nonlinearity coefficient assessment methods, the presented technique works in the pulse-echo mode and it doesn't require information on second harmonic or using a special wide-band transducer. The method can be implemented directly into a medical scanner.

Plane wave imaging, nonlinear ultrasound, quantitative ultrasound, coefficient of nonlinearity

Nakagami distribution is used to model the statistical properties of backscattered echoes in tissue. The proper estimate requires the compensation of attenuation along each scanning line. Attenuation of the wave results in decreasing of the envelope mean intensity with depth what modifies the Nakagami scale parameter. This phenomenon violates the assumption that envelope samples within region of interest are identically distributed and disrupts estimation. Here, we investigate the influence of wave attenuation on Nakagami shape parameter estimators for various scattering scenarios, attenuation coefficients and region of interest size. Three methods are proposed to solve this issue. Scans of a thyroid and of a breast lesion are analyzed. It was found that proposed methods improved the estimation, especially when larger regions were used to collect envelope samples.

ultrasound, breast cancer, Nakagami distribution

The aim of the study was finding the relationship between BIRADS classification combined with envelope K and Nakagami statistics of the echoes backscattered in the breast tissue in vivo and the histological data. 107 breast lesions were examined. Both, the RF echo-signal and B-mode images from the lesions and surrounding tissue were recorded. The analysis method was based on the combining data from BIRADS classifications and both distributions parameters. 107 breasts lesions - 32 malignant and 75 benign - were examined. When only BIRADS classification was used all malignant lesions were diagnosed correctly, however 34 benign lesions were sent for the biopsy unnecessarily. For K distribution the sensitivity and specificity were 78.13%, and 86.67% while for Nakagami statistics the sensitivity and specificity were 62.50% and 93.33%, respectively. Combined K and BIRADS resulted in sensitivity of 96.67% and specificity 60%. Combined BIRADS (3/4a cut-off) plus Nakagami statistics showed 100% of sensitivity with specificity equal 57.33%, decreasing the number of lesions which were biopsied from 34 to 28.

breast cancer, quantitative ultrasound, BIRADS

Flow Mediated Dilation (FMD) is a technique widely used to assess the endothelial function by ultrasound. Ideally, both the brachial artery wall shear stress (stimulus) and the diameter change (effect) shall be estimated and monitored for up to 10 minutes, while blood flow is restricted by a cuff and then suddenly released. An inherent method's difficulty is maintaining the linear array probe aligned with the artery for such a long time. The problem is here faced by an integrated hardware/software approach that displays in real-time both the spatial velocity profiles and the diameter changes, and acquires raw data all over the exam.

component, Flow mediated dilation, FMD, wall shear stress, wall shear rate, diameter distension, ULA-OP

Nowadays there are new modalities in ultrasound imaging allowing better characterization of tissue regions with different stiffness. We are proposing an approach based on simultaneous propagation of two waves being a combination of two pulses differing in pressure and frequency: a low frequency pulse is expected to change the local scattering properties of the tissue due to compression/rarefaction while a high frequency pulse is used for imaging. Two transmissions are performed for each scanning line. First, with the imaging pulse that propagates on maximum compression caused by a low frequency wave. Next, the low frequency wave is inverted and the imaging pulse propagates over the maximum rarefaction. After the processing of the subtracted echoes from subsequent transmissions including wavelet transform and band-pass filtering, differential images were reconstructed. The low frequency wave has a visible impact on the scattering properties of the tissue which can be observed on a differential image.

Breast cancer is the most common cancer in women, and ultrasound
(US) imaging is important for breast mass assessment. Accurate automatic
breast mass segmentation facilitates mass characterization. Traditional deep
learning methods, such as convolutional networks and transformers, have
achieved high performance in breast mass segmentation. Recently, implicit
neural representations, which use continuous, nonlinear, coordinate-based
approximations through multi-layer perceptrons, have shown promise
in various fields, including medical image segmentation. In this work, we
present an implicit network for breast mass segmentation in US. We train a
coordinate-based implicit network to jointly output the US image pixel values
and the segmentation pixel scores. The network is conditioned using latent
codes, effectively associating the regression and segmentation tasks with the
mass type (benign/malignant) and BI-RADS category. Additionally, a trainable
image-specific code is used. During inference, given a US image, we fix the
weights of the network and use the backpropagation algorithm to determine
the latent codes, facilitating the image regression task. This process, due to the learned associations, also provides the segmentation mask. Our results
confirm the feasibility of using implicit networks for breast mass segmentation
and other tasks leveraging learned associations between latent codes and
image/mask appearances.

We developed breast mass classification methods based on deep convolutional neural networks (CNNs) and morphological features (MF), then compared those to assessment of four experienced radiologists employing BI-RADS protocol. The classification models were developed based on 882 clinical ultrasound B-mode images of masses with confirmed findings and regions of interest indicating mass areas. Various transfer learning techniques, including fine-tuning of a pre-trained CNN, were investigated to develop deep learning models. A matching layer technique was applied to convert gray-scale images to red, green, blue to efficiently utilize discrimination of the pre-trained model. For the classical approach, we calculated MF related to breast mass shape (e.g., height-width ratio, circularity) and then trained binary classifiers. We additionally evaluated both approaches using two publicly available US datasets. Several statistical measures (area under the receiver operating curve [AUC], sensitivity and specificity) were used to assess the classification performance on a test set of 150 cases. The matching layer significantly increased AUC from 0.895 to 0.936 while radiologists’ AUCs ranged from 0.806 to 0.882. This study shows both deep learning and classical models achieve high performance. When developed as a clinical tool, the methods examined in this study have potential to aid radiologists accurate breast mass classification with ultrasound.

Qualitative sonography is used to assess nonalcoholic fatty liver disease (NAFLD), an important health issue worldwide. We used B-mode image deep-learning to objectively assess NAFLD in 4 views of the liver (hepatic veins at confluence with inferior vena cava, right portal vein, right posterior portal vein and liver/kidney) in 135 patients with known or suspected NAFLD. Transfer learning with a deep convolutional neural network (CNN) was applied for quantifying fat fraction and diagnosing fatty liver (≥ 5%) using contemporaneous MRI-PDFF as ground truth. Single and multi-view learning approaches were compared. Class activation mapping generated attention maps to highlight regions important for deep learning-based recognition. The most accurate single view was hepatic veins, with area under the receiver operating characteristic curve (AUC) of 0.86 and Spearman’s rank correlation coefficient of 0.65. A multi-view ensemble of deep-learning models trained for each view separately improved AUC (0.93) and correlation coefficient (0.76). Attention maps highlighted regions known to be used by radiologists in their qualitative assessment, e.g., hepatic vein-parenchyma interface and liver-kidney interface. Machine learning of four liver views can automatically and objectively assess liver fat. Class activation mapping suggests that the CNN focuses on similar features as radiologists. [No. R01DK106419.]

We investigated the use of high-frequency quantitative ultrasound (QUS) and B-mode texture features to characterize ulnar and median nerve fascicles using a clinical scanner (Vevo MD) and a 30-MHz center-frequency probe. US correlation with histology was first investigated in the ulnar nerve in situ in cadaveric specimens. 85 fascicles were matched in B-mode images and the histology sections. Collagen and myelin concentrations were quantified from trichrome labeling, and backscatter coefficient (-24.89 ± 8.31 dB), attenuation coefficient (0.92 ± 0.04 dB/cm MHz), Nakagami parameter (1.01 ± 0.18) and entropy (6.92 ± 0.83) were calculated from ultrasound data. B-mode texture features were obtained via the gray-level co-occurrence matrix algorithm. Combined collagen and myelin concentration were significantly correlated with the backscatter coefficient (R = -0.68), entropy (R = -0.51), and several texture features. For the median nerve, we measured backscatter and morphology in 10 patients with carpal tunnel syndrome and 21 healthy volunteers. Significant differences (<0.01) between patients and controls and AUC 0.89–0.94 for QUS biomarkers were observed. Our study indicates that QUS may potentially provide useful information on structural components of even very small nerves (2 × 4 mm) and fascicles for diagnosing and monitoring injury, and surgical planning.

The ultimate goal of this work is a real-time processing framework for ultrasound image reconstruction augmented with machine learning. To attain this, we have implemented WaveFlow – a set of ultrasound data acquisition and processing tools for TensorFlow. WaveFlow includes: ultrasound Environments (connection points between the input raw ultrasound data source and TensorFlow) and signal processing Operators (ops) library. Raw data can be processed in real-time using algorithms available both in TensorFlow and WaveFlow. Currently, WaveFlow provides ops for B-mode image econstruction (beamforming), signal processing and quantitative ultrasound. The ops were implemented both for the CPU and GPU, as well as for built-in automated tests and benchmarks. To demonstrate WaveFlow’s performance, ultrasound data were acquired from wire and cyst phantoms and elaborated using selected sequences of the ops. We implemented and valuated: Delay-and-Sum beamformer, synthetic transmit aperture imaging (STAI), planewave imaging (PWI), envelope detection algorithm and dynamic range clipping. The benchmarks were executed on the NVidiaR Titan X GPU integrated in the USPlatform research scanner (us4us Ltd., Poland). We achieved B-mode image reconstruction frame rates of 55 fps, 17 fps for the STAI and the PWI algorithms, respectively. The results showed the feasibility of realtime ultrasound image reconstruction using WaveFlow operatorsin the TensorFlow framework. WaveFlow source code can be found at github.com/waveflow-team/waveflow.

Poly(vinyl alcohol) cryogel, PVA-C, is produced as a soft tissue-mimicking material, suitable for application in ultrasound imaging. A 10% by weight poly(vinyl alcohol) in water solution was used to form PVA-C, which is solidified through a freeze–thaw process. The number of freeze–thaw cycles affects the properties of the material, particularly the mechanical stiffness. The ultrasound characteristics were investigated using 3 different cylindrical samples of PVA-C produced by 1, 2 and 3 cycles of freezing-thawing process. The speed of sound was found to range from 1502 to 1522 m s−1, and the attenuation coefficients were in the range of 0.085–0.124 dB/(cm MHz). The structural eterogeneities are visualized by Nakagami maps and it is shown that the range of Nakagami parameter characterize the differences between samples. The samples are structurally different in the regions close to the surface from the internal regions. This is probably caused by the spatial heterogeneity of the solidification process. The thickness of the boundary layer is also measured from Nakagami maps and it is shown that it is also linked to the type of samples. The elastography maps (measured by the commercial quasistatic strain imaging system …) are compared with Nakagami maps. The question arises, in what circumstances parametric estimation of spatial structure variations by Nakagami maps are linked to the spatial variations of local stiffness?

soft tissue phantoms, elastography, ultrasound attenuation, speed of sound, Nakagami maps, stiffness

soft tissue phantoms, backscattered ultrasonic signal, changes in the backscattered energy

The practical aim of this research is to detect the temperature by the selected properties of the backscattered ultrasound signals collected during heating/cooling of the soft tissue sample. The initial data are the raw backscattered signals, RF (radio frequency) signals, which form the two-dimensional matrix. These data are divided according to the regions of interest (ROI) analyzed piece-wise in the following way:
• absolute value of Hilbert transform in each time sample is calculated,
• the approximations with Daubeschies 6 wavelets is performed,
• Kolmogorov-Smirnov distance and Kullback-Laibler divergence between initial ROI statistics and the statistics of the ROI in succesive temperature level are used to visualization of the dynamic temperature changes on the map of the sample volume.

temperaturę detection, non-parametric statistics, backscattered ultrasound, wavelet

A B-mode ultrasonography provides structural information on the tissue under investigation encoding the echo strength in gray scale in a two-dimensional image. Interpretation of the B-mode image of breast tissue is done by a physician. The analysis of statistical properties of backscattered RF signal has been recently applied successfully to distinct healthy tissue from tissue lesions regions as a new method of quantitative ultrasound (QUS). Up till now, the most reliable results were obtained for liver and renal tissue lesions, because their normal, healthy structures are nearly homogeneous while a heterogeneous breast tissue classification is still an open issue. The recent study revealed that the medium contraction and expansion induced by a temperature change may cause variations in the relative position of scatterers in a tissue. We have developed a new procedure of heating the patient breast and allowing to observe and record in vivo the influence of temperature changes on a B-mode image and properties of unprocessed radio frequency (RF) backscattered echoes. The initial, feasibility studies of influence of the temperature increase in breast tissue on the intensity, spectrum and statistics of ultrasonic echoes will be discussed.

breast tissue, RF signal, backscattered signal amplitude statistics, spectral properties

To target a focal spot of an ultrasound beam on a tumor located deep inside tissues during thermo-ablative treatment by HIFU technique, beams with different focal distances are required. To be able to control a depth of local thermal fields induced in tissues by a single beam, both, the planar and concave 7-element annular phased array transducers with a 2 MHz frequency and 29 mm diameter generating beams with electronically controlled focal length were designed and produced. The radius of curvature (ROC) for the concave transducer was equal to 60 mm. Elements of each transducer had the same area to provide uniform pressure distribution on the radiating surface due to the same impedance and were excited by pulses with time delays providing the beam focusing in water at three different depths (25 mm, 30 mm, and 35 mm). To select sets of time delays for each focal depth the measurements of pressure waveforms on the axis of each beam generated in water were performed using a needle hydrophone. For these measurements 10-cycle tone bursts with 1 kHz PRF were used. In order to induce local thermo-ablative necrosis inside a tissue at three different depths (10mm, 15mm, and 20mm) a two-layer media of propagation comprising of 15-mm layer of water and 25-mm layer of pork loin was used. To heat the pork loin locally 20-cycle tone bursts with 0.2 duty-cycle and average acoustic power varied between12W and 18W (initial intensity ISATA varied between 2W/cm2 and 3W/cm2) was applied. In order to determine the exposure time required to induce necrosis (rise in temperature to 56 °C) inside the pork loin sample at the selected depth the thermocouples placed on the acoustic beam axis were used. After exposure to focused ultrasound three necrotic lesions were observed after cutting the tested tissue sample along the axes of the beams used. The obtained results proved the feasibility of controlling the depth of local tissue necrosis using pulsed focused ultrasound beams with electronically movable focal spot generated by the annular phased array transducer designed.

annular phased array transducer, pulsed High Intensity Focused Ultrasound, electronically movable focus, local tissue heating, thermal ablation, tissue necrosis