Institute of Fundamental Technological Research

Phased-Array Ultrasonic Technique is traditionally used for the non-destructive inspection of welds and supported by industrial-grade inspection equipment. FullMatrix Capture (FMC) with Total Focusing Method (TFM) provide new capabilities and multimodal imaging, but available commercial scanners have limitations in acquisition speed (30–300MB/s) and reconstruction speed. Our goal was to develop a solution for FMC acquisition that can be applied to high-speed robotized weld scanning (speed of 100 mm/s with a resolution of 1 mm). For FMC acquisition, we have applied a portable programmable ultrasound research system us4R-lite™ (us4us Ltd., Poland) in a 64:256 channel configuration and standard angled 32-element Phased-Array probes. The system can acquire and store raw RF or demodulated I/Q data at a speed of 2–6 GB/s, enabling real-time FMC at high speed. Data can be stored on a PC during scanning and processed by a high-performance GPU. We have successfully tested our experimental setup while scanning flat-section welds with a motorized scanner at a speed approaching 100 mm/s. The acquisition and processing software developed uses Nvidia CUDA on GPU and can manage real-time storage and scanning. Next, we are planning to integrate the solution into an industrialgrade high-speed FMC acquisition system with embedded GPU processing.

Ultrasonic Testing (UT) (4285), robotic inspection (23), PAUT (42), FMC (16), TFM (28), GPU processing

In this paper, we propose a novel deep learning method for joint classification and segmentation of breast masses based on radio-frequency (RF) ultrasound (US) data. In comparison to commonly used classification and segmentation techniques, utilizing B-mode US images, we train the network with RF data (data before envelope detection and dynamic compression), which are considered to include more information on tissue’s physical properties than standard B-mode US images. Our multi-task network, based on the Y-Net architecture, can effectively process large matrices of RF data by mixing 1D and 2D convolutional filters. We use data collected from 273 breast masses to compare the performance of networks trained with RF data and US images. The multi-task model developed based on the RF data achieved good classification performance, with area under the receiver operating characteristic curve (AUC) of 0.90. The network based on the US images achieved AUC of 0.87. In the case of the segmentation, we obtained mean Dice scores of 0.64 and 0.60 for the approaches utilizing US images and RF data, respectively. Moreover, the interpretability of the networks was studied using class activation mapping technique and by filter weights visualizations.

breast mass classification, breast mass segmentation, convolutional neural networks, deep learning, quantitative ultrasound, ultrasound imagin

In this work, we propose a deep learning method for breast mass segmentation in ultrasound (US). Variations in breast mass size and image characteristics make the automatic segmentation difficult. To addressthis issue, we developed a selective kernel (SK) U-Net convolutional neural network. The aim of the SKswas to adjust network's receptive fields via an attention mechanism, and fuse feature maps extractedwith dilated and conventional convolutions. The proposed method was developed and evaluated usingUS images collected from 882 breast masses. Moreover, we used three datasets of US images collectedat different medical centers for testing (893 US images). On our test set of 150 US images, the SK-U-Netachieved mean Dice score of 0.826, and outperformed regular U-Net, Dice score of 0.778. When evaluatedon three separate datasets, the proposed method yielded mean Dice scores ranging from 0.646 to 0.780. Additional fine-tuning of our better-performing model with data collected at different centers improvedmean Dice scores by ~6%. SK-U-Net utilized both dilated and regular convolutions to process US images. We found strong correlation, Spearman's rank coefficient of 0.7, between the utilization of dilated convo-lutions and breast mass size in the case of network's expansion path. Our study shows the usefulness ofdeep learning methods for breast mass segmentation. SK-U-Net implementation and pre-trained weightscan be found at github.com/mbyr/bus_seg.

attention mechanism, breast mass segmentation, convolutional neural networks, deep learning, receptive field, ultrasound imaging

We propose a novel approach to breast mass classification based on deep learning models that utilize raw radio-frequency (RF) ultrasound (US) signals. US images, typically displayed by US scanners and used to develop computer-aided diagnosis systems, are reconstructed using raw RF data. However, information related to physical properties of tissues present in RF signals is partially lost due to the irreversible compression necessary to make raw data readable to the human eye. To utilize the information present in raw US data, we develop deep learning models that can automatically process small 2D patches of RF signals and their amplitude samples. We compare our approach with classification method based on the Nakagami parameter, a widely used quantitative US technique utilizing RF data amplitude samples. Our better performing deep learning model, trained using RF signals and their envelope samples, achieved good classification performance, with the area under the receiver attaining operating characteristic curve (AUC) and balanced accuracy of 0.772 and 0.710, respectively. The proposed method significantly outperformed the Nakagami parameter-based classifier, which achieved AUC and accuracy of 0.64 and 0.611, respectively. The developed deep learning models were used to generate parametric maps illustrating the level of mass malignancy. Our study presents the feasibility of using RF data for the development of deep learning breast mass classification models.

breast lesion classification, convolutional neural networks, deep learning, radio-frequency signals, ultrasound imaging

Ultrasound (US) imaging is widely used for the tissue characterization. However, US images commonly suffer from speckle noise, which degrades perceived image quality. Various deep learning approaches have been proposed for US image denoising, but most of them lack the interpretability of how the network is processing the US image (black box problem). In this work, we utilize a deep reinforcement learning (RL) approach, the pixelRL, to US image denoising. The technique utilizes a set of easily interpretable and commonly used filtering operations applied in a pixel-wise manner. In RL, software agents act in an unknown environment and receive appropriate numerical rewards. In our case, each pixel of the input US image has an agent and state of the environment is the current US image. Agents iteratively denoise the US image by executing the following pixel-wise pre-defined actions: Gaussian, bilateral, median and box filtering, pixel value increment/decrement and no action. The proposed approach can be used to generate action maps depicting operations applied to process different parts of the US image. Agents were pre-trained on natural gray-scale images and evaluated on the breast mass US images. To enable the evaluation, we artificially corrupted the US images with noise. Compared with the reference (noise free US images), filtration of the images with the proposed method increased the average peak signal-to-noise ratio (PSNR) score from 14 dB to 26 dB and increased the structure similarity index score from 0.22 to 0.54. Our work confirms that it is feasible to use pixel-wise RL techniques for the US image denoising.

deep reinforcement learning, ultrasound imaging, image denoising, filtration, breast cancer

In this work, we considered the implementation of a 3D volume reconstruction algorithm for single plane-wave ultrasound insonification. We review the theory behind the Hybrid Spectral-Domain Imaging (HSDI) algorithm, provide details of the algorithm implementation for Nvidia CUDA GPU cards, and discuss the performance evaluation results. The average time required to reconstruct a single data volume using our GPU implementation of the HSDI algorithm was 22 ms. We also present an iso-surface extraction result using a marching cubes algorithm. Our work constitutes a preliminary research for further development and implementation of 3D volume reconstruction using GPU implementation of the spectral domain imaging algorithm.

ultrasound imaging, 3D ultrasound, volumetric imaging, gpu

Ultrasound is becoming the modality of choice for everyday medical diagnosis, due to its mobility and decreasing price. As the availability of ultrasound diagnostic devices for untrained users grows, appropriate guidance becomes desirable. This kind of support could be provided by a software agent, who easily adapts to new conditions, and whose role is to instruct the user on how to obtain optimal settings of the imaging system during an examination. In this work, we verified the feasibility of implementing and training such an agent for ultrasound, taking the deep reinforcement learning approach. The tasks it was given were to find the optimal position of the transducer’s focal point (FP task) and to find an appropriate scanning plane (PP task). The ultrasound environment consisted of a linear-array transducer acquiring information from a tissue phantom with cysts forming an object-of-interest (OOI). The environment was simulated in the Field-II software. The agent could perform the following actions: move the position of the probe to the left/right, move focal depth upwards/downwards, rotate the probe clockwise/counter-clockwise, or do not move. Additional noise was applied to the current probe setting. The only observations the agent received were B-mode frames. The agent acted according to stochastic policy modeled by a deep convolutional neural network, and was trained using the vanilla policy gradient update algorithm. After the training, the agent’s ability to accurately locate the position of the focal depth and scanning plane improved. Our preliminary results confirmed that deep reinforcement learning can be applied to the ultrasound environment.

ultrasound guidance, reinforcement learning, deep learning

Ultrasound medical diagnostics is a real-time modality based on a doctor's interpretation of images. So far, automated Computer-Aided Diagnostic tools were not widely applied to ultrasound imaging. The emerging methods in Artificial Intelligence, namely deep learning, gave rise to new applications in medical imaging modalities. The work's objective was to show the feasibility of implementing deep learning algorithms directly on a research scanner with GPU software beamforming. We have implemented and evaluated two deep neural network architectures as part of the signal processing pipeline on the ultrasound research platform USPlatform (us4us Ltd., Poland). The USPlatform is equipped with a GPU cluster, enabling full software-based channel data processing as well as the integration of open source Deep Learning frameworks. The first neural model (S-4-2) is a classical convolutional network for one-class classification of baby body parts. We propose a simple 6-layer network for this task. The model was trained and evaluated on a dataset consisting of 786 ultrasound images of a fetal training phantom. The second model (Gu-net) is a fully convolutional neural network for brachial plexus localisation. The model uses 'U-net'-like architecture to compute the overall probability of target detection and the probability mask of possible target locations. The model was trained and evaluated on 5640 ultrasound B-mode frames. Both training and inference were performed on a multi-GPU (Nvidia Titan X) cluster integrated with the platform. As performance metrics we used: accuracy as a percentage of correct answers in classification, dice coefficient for object detection, and mean and std. dev. of a model's response time. The 'S-4-2' model achieved 96% classification accuracy and a response time of 3 ms (334 predictions/s). This simple model makes accurate predictions in a short time. The 'Gu-net' model achieved a 0.64 dice coefficient for object detection and a 76% target's presence classification accuracy with a response time of 15 ms (65 predictions/s). The brachial plexus detection task is more challenging and requires more effort to find the right solution. The results show that deep learning methods can be successfully applied to ultrasound image analysis and integrated on a single advanced research platform

Ultrasonic imaging, Neural networks, Convolution, Machine learning, Image segmentation, Kernel

In this work, we present a low-cost, portable, and fully configurable ultrasound system implementing 2-D real-time Shear Wave Elastography (SWE) imaging mode. To achieve that we have enhanced the transmit capabilities of the 256 TX/64 RX us4R-lite research system, developed by our team, to support push pulses generation. This system was combined with a signal processing pipeline reconstructing stiffness maps from raw RF data. Real-time imaging performance was provided by an efficient reconstruction algorithm execution that incorporated graphics processing unit (GPU). The overall system performance was assessed experimentally using an industry-standard elasticity Q/A phantom. Relevant reconstruction parameters were evaluated in terms of reconstruction time. The system achieved stiffness estimation with a bias <5% and SNR of 30 dB and was able to detect lesions of size >4 mm and various stiffness with CNR in the range of 13–17 dB. The system throughput of up to 5 fps has been achieved on a PC notebook equipped with NVIDIA RTX 3060 GPU.

Ab-initio, atomistic modeling, tensor fields, dislocation fields algebra, lattice distortions, visualization methods

The uNET neural network architecture has shown very promising results when applied to semantic segmentation of biomedical images. The aim of this work is to check whether this architecture is equally applicable to semantic segmentation distinguishing the structural elements of railway viaducts. Artificial images generated by a computer graphics program rendering the 3D model of the viaduct in a photorealistic manner will be used as data sets. This approach produces a large number of
images that provide a solid training set for machine learning model.

Computer vision, deep learning, semantic segmentation

The ultimate goal of this work is a real-time processing framework for ultrasound image reconstruction augmented with machine learning. To attain this, we have implemented WaveFlow – a set of ultrasound data acquisition and processing tools for TensorFlow. WaveFlow includes: ultrasound Environments (connection points between the input raw ultrasound data source and TensorFlow) and signal processing Operators (ops) library. Raw data can be processed in real-time using algorithms available both in TensorFlow and WaveFlow. Currently, WaveFlow provides ops for B-mode image econstruction (beamforming), signal processing and quantitative ultrasound. The ops were implemented both for the CPU and GPU, as well as for built-in automated tests and benchmarks. To demonstrate WaveFlow’s performance, ultrasound data were acquired from wire and cyst phantoms and elaborated using selected sequences of the ops. We implemented and valuated: Delay-and-Sum beamformer, synthetic transmit aperture imaging (STAI), planewave imaging (PWI), envelope detection algorithm and dynamic range clipping. The benchmarks were executed on the NVidiaR Titan X GPU integrated in the USPlatform research scanner (us4us Ltd., Poland). We achieved B-mode image reconstruction frame rates of 55 fps, 17 fps for the STAI and the PWI algorithms, respectively. The results showed the feasibility of realtime ultrasound image reconstruction using WaveFlow operatorsin the TensorFlow framework. WaveFlow source code can be found at github.com/waveflow-team/waveflow.