Institute of Fundamental Technological Research

A new detailed dataset of breast ultrasound scans (BrEaST) containing images of benign and malignant lesions as well as normal tissue examples, is presented. The dataset consists of 256 breast scans collected from 256 patients. Each scan was manually annotated and labeled by a radiologist experienced in breast ultrasound examination. In particular, each tumor was identified in the image using a freehand annotation and labeled according to BIRADS features and lexicon. The histopathological classification of the tumor was also provided for patients who underwent a biopsy.
The BrEaST dataset is the first breast ultrasound dataset containing patient-level labels, image-level annotations, and tumor-level labels with all cases confirmed by follow-up care or core needle biopsy result. To enable research into breast disease detection, tumor segmentation and classification, the BrEaST dataset is made publicly available with the CC-BY 4.0 license.

Manual image segmentations are naturally subject to inaccuracies related to systematic errors (due to the tools used, eye-hand coordination, etc.). This was noted earlier when a simplified accuracy scale was proposed [1]. This scale arbitrarily divides a given range of values of the Kappa measurement parameter into classes: almost perfect (>0.80), substantial (0.61 - 0.80), moderate (0.41 - 0.60), fair (0.21 - 0.40), slight (0.00 - 0.21) and poor (< 0.00). However, the determination of threshold values between classes is not entirely clear and seems to be application-dependent. This is particularly important for images in which the tumor-normal tissue boundary can be very indistinct, as is observed in ultrasound imaging of the most common cancer in women - breast cancer [2]. In machine learning, there is an ongoing contest over the values of performance indicators obtained from new neural network architecture without accounting for any ground truth bias. This raises the question of what relevance, from a segmentation quality point of view, a gain at the level of single percentages has [3] if the references have much greater uncertainty. So far, research on this topic has been limited. The relationship between the segmentations of breast tumors on ultrasound images provided by three radiologists and those obtained using deep learning model has been studied in [4]. Unfortunately, the indicated segmentation contour sometimes varied widely in all three cases. A cursory analysis by multiple physicians, which focused only on the Kappa coefficient in the context of physicians’ BI-RADS category assignments, was conducted in the [5]. In this article, we present a preliminary analysis of the accuracy of experts’ manually prepared binary breast cancer masks on ultrasound images and their impact on performance metrics commonly used in machine learning. In addition, we examined how tumor type or BI-RADS category [6] affects the accuracy of tumor contouring.