Deep feature learning for soft tissue sarcoma classification in MR images via transfer learning

Medical image analysis is motivated by deep learning emergence and computation power increase. Meanwhile, relevant deep features can significantly enhance learnable expert and intelligent systems performance and reduce diagnosis time and arduousness. This paper presents a deep learning-based radiomics framework for aided diagnosis of soft tissue sarcomas of the extremities. MR Images with histologically confirmed Liposarcoma (LPS) and Leiomyosarcomas (LMS) have been retrieved from the Cancer Imaging Archives database and pre-processed to recuperate ROIs from MR scans with delineated tumors. This study investigates the significance and impact of medical image fusion on deep feature learning based on transfer learning from the natural domain to the medical domain. Towards this end, we propose to fuse T1 with T2FS or STIR modalities using type-2 fuzzy sets in the non-subsampled shearlet domain. Being decomposed, low-frequency sub-images were selected using local energy and type-2 fuzzy entropy, while high frequencies were selected according to the maximum of the absolute value. Experimental results indicated that the proposed fusion framework outperformed the state-of-the-art fuzzy logic-based fusion techniques in terms of entropy and mutual information. Accordingly, we fine-tuned the pre-trained AlexNet deep convolutional neural network (CNN) with stochastic gradient descent (SGD). First, with the pre-processed dataset, and second with the fused images. As a result, the average classification accuracy using the augmented training data by image rotation and flipping was 97.17% with the raw data and 98.28% with the fused images, which highlighted the usefulness of complementary information for deep feature learning. One crucial concern was to investigate the depth of knowledge transferability. We incrementally fine-tuned the pre-trained CNN to assess the required level that achieves performance improvements in STS classification. Through layer-wise fine-tuning, our study further confirms the potential of middle and deep layers in performance improvement. Moreover, the transferability was concluded better than random weights. With the encouragement of classification results, our aided diagnosis framework may be in the pipeline to assist radiologists in classifying LPS and LMS.