Application of MRI Radiomics Combined With Deep Learning Technology in Glioma Segmentation and Survival Prognosis

I.INTRODUCTION

Brain tumors are abnormal cells that grow in the brain and can generally be divided into two categories, primary brain tumors and secondary brain tumors, according to the source of the abnormal cells. Primary brain tumors are cells that originate directly from abnormal growths in the brain tissue, while secondary brain tumors are abnormal cells that spread to the brain after normal cells in other parts of the body have become cancerous. Magnetic resonance imaging (MRI), as a typical noninvasive imaging technique, can produce high-quality brain images without damage and cranial artifacts, which can provide more comprehensive information for the diagnosis and treatment of brain tumors, and is the main technical means for the diagnosis and treatment of brain tumors. The segmentation of brain tumor image by MRI is a very important step in the process of diagnosis and treatment of brain tumors, and with the help of multimodal brain image segmentation of tumors, doctors can perform quantitative analysis from the brain tumor thereby measuring the maximum diameter, volume, and amount of diseased tissue in the brain and develop the best diagnosis and treatment plan for the patient to quantify the response of the brain tumor before and after treatment.

Due to the needs of clinical application and scientific research, MRI brain tumor image segmentation has also become an important component in the field of medical imaging and has received extensive attention for a long time, and many segmentation methods for MRI brain tumor images have been proposed by researchers, but the existing methods are still not able to achieve satisfactory results in clinical MRI brain image segmentation all the time, which is also determined by the complexity of MRI brain images. In general, the difficulties in MRI brain image segmentation can be summarized as follows: 1) the limitations of imaging technology and image acquisition process factors lead to the existence of field shift effects, volume effects, motion artifacts, and noise in MRI brain images; 2) the tissue structure of the brain is very complex and not easy to be distinguished, and MRI brain images include not only the normal tissues of the brain such as the cortex, the cerebrospinal fluid, the white matter, the grey matter, and other normal tissues but also enhanced tumors, non-enhanced tumors, non-enhanced tumors, and other normal tissues, and also enhanced tumors, including enhanced tumors, non-enhanced tumors, edema, and necrotic areas within the tumor and other lesion tissues, and some of the tissue structures have high imaging similarity and are not easy to be distinguished; 3) the spatial location, shape, and size of brain tumors are variable, and the boundaries between different soft tissues or between soft tissues and lesion tissues may be blurred; 4) there are large differences in the structure of normal tissues and lesion tissues between different individuals, and even the same individual at different times may also have large differences. These factors have a significant impact on feature extraction and accurate segmentation in brain tumor image segmentation, and in the actual segmentation process, the impact of these factors can be handled accurately and timely to ensure the accuracy and robustness of brain tumor image segmentation.

Glioma is one of the most common malignant primary brain tumors in adults [1]. Gliomas are classified as grades I–IV based on malignancy, with different prognoses for each grade. Glioblastomas, the most aggressive type of brain tumor, account for roughly half of newly diagnosed gliomas [2,3,4]. In concurrent clinical trials, the median patient survival time for glioblastoma was approximately 14–17 months [5,6], compared to 12 months in population-based studies [7], with a 5-year survival rate of only 5.6% [1]. Early and accurate MRI evaluation of brain tumors affects patients’ estimated survival time and directly impacts patient treatment planning and tumor growth assessment. It is essential for improving patient care and survival rates [8,9].

Artificial intelligence has recently accelerated the development of several disciplines, including medicine. Several methods based on machine learning and deep learning to assess the prognosis of oncology patients have been rapidly developed [10,11]. Artificial intelligence methods for glioma have been developed, including predictive imaging grading, imaging genomics, and prognosis prediction [8,10,12]. Ahammed Muneer K. V. et al. presented a method for automatic glioma tumor grading from MRI images using a classifier from the Wndchrm tool and the VGG-19 deep convolutional neural network (DNN). The results showed that the classification accuracy of the Wndchrm classifier was 92.86%, and the classification accuracy of the VGG-19 DNN was 98.25% [13]. Many studies were based on the 3D-Uet algorithm. The segmentation performance of one of these studies classified gliomas with a dice score of 90% and an AUC of the glioma grading model of 94% in the validation set [14], while another study classified gliomas with a dice score of 91% [15]. Huigao Luo. et al. used this algorithm to diagnose and prognosticate gliomas, achieving 80.4% and 83.9% accuracy [16]. Similarly, the COX regression prediction model was also used with a C-index of 0.710 for OS prediction [17].

However, due to the small number of medically relevant datasets involved, there are still few automatic segmentation and prognostic assessment models based on deep learning radiomics, and the accuracy of the models requires further improvement. Manual segmentation of a clinically large number of MRI images for cancer diagnosis is a complex and time-consuming task, so brain tumor image automatic segmentation is required. Deep learning methods can efficiently process large amounts of MRI image data. Therefore, in this study, we propose an automatic segmentation network based on a deep 3D convolutional neural network and a new radiomics model with depth features to predict overall survival in glioblastoma patients. The patient’s lesion area is obtained based on the automatic segmentation technique, and radiomic features are extracted using pyradiomics with Python 3.7. After combining other clinical features, the features with a correlation greater than 0.7 are screened using the preprocessing Pearson correlation coefficient method. Then t-test is used to remove redundant features, and a lasso is used to reduce the dimensionality to obtain the important features. In addition, nine machine learning methods, including support vector machine, random forest, and XGBoost, are used to predict the overall survival and prognosis of glioma patients. Finally, the advantages and disadvantages of these algorithms are successfully compared. This study may provide a clinical interpretation of GBM classification and help us better to understand the relationship between glioblastoma and radiomic features.

Our primary contributions in this study can be summarized as follows:

• •A novel hybrid model combines deep learning-based segmentation with radiomics and clinical features for survival prediction is proposed for medical image segmentation.
• •An improved feature extraction pipeline that integrates deep learning features with traditional radiomics features is designed to capture both high-level and low-level imaging patterns, which are critical for accurate glioma segmentation and survival prediction.
• •Advanced data preprocessing techniques, such as intensity normalization and adaptive contrast enhancement, are utilized to reduce data heterogeneity and improve the quality of MRI scans.
• •Our method aimed to be clinically applicable, with a focus on automating the segmentation and prognosis processes. By reducing the reliance on manual segmentation, our approach can save time and minimize inter-observer variability, which is a significant challenge in clinical practice.

The rest of the paper is organized as follows. Section II discusses related studies. Section III illustrates the proposed method. Section IV introduces the experiments and results. Section V presents discussion and conclusions.

II.RELATED WORKS

Deep neural network models represented by AlexNet, VGGNet, ResNet, DenseNet, and GAN have been successfully applied to many computer vision tasks and have gained widespread attention in academia and industry. Given the powerful automatic extraction of highly discriminative features demonstrated by deep neural networks, they have been rapidly applied to the field of medical image processing and analysis. Meanwhile, deep learning-based brain tumor computer-aided diagnosis research has also gained widespread attention, especially the International Association for Medical Image Computing and Computer-Assisted Intervention (MICCAI) has organized the Multimodal Brain Tumor Segmentation Competition for consecutive years since 2012, which has greatly promoted the development of deep learning-based MRI brain tumor image segmentation research. As deep learning automatically acquires MRI tumor feature information from the training sample set through an end-to-end learning approach, it changes the manual feature extraction of traditional methods and achieves breakthrough performance progress over traditional methods. Based on the use of sample data labeling during network training, deep neural networks can be classified into supervised, semi-supervised, and unsupervised deep neural networks. Since supervised networks are able to make full use of data class label information during training, they generally achieve better accuracy performance than semi-supervised and unsupervised networks. In the research of deep learning methods for MRI brain tumor image segmentation, most of the existing deep segmentation networks also adopt supervised learning and have effectively explored various aspects such as network architecture, global or local contextual information feature extraction, and multi-scale feature fusion. In addition, due to the relatively small size of finely labeled brain tumor samples, some scholars have also carried out research on semi-supervised and unsupervised MRI brain tumor segmentation networks and obtained effective progress. Jianwu Long et al. [18] proposed a feature similarity module and a reliable region enhancement module to utilize the dense feature information of unlabeled data by combining dense feature prediction and segmentation features. This method pays more attention to the information utilization at the feature level and reduces pseudo-label noise by enhancing the weights of reliable regions. Lin Wei et al. [19] designed a novel semi-supervised learning framework that is the dual consistency regularization network for medical image segmentation. Model-level consistency effectively reduces the noise interference of pseudo-labels, while task-level consistency fully utilizes multi-scale information. The combination of these two aims to approximate low-entropy decision boundaries.

In recent years, deep learning radiomics has been increasingly used in gliomas; however, there is still room to explore the depth characteristics and their prognosis and grading. Most of the research focuses on using deep learning networks in tumor grading and classification [20,21]. For example, Hiba Mzoughi et al. proposed an efficient and fully automated deep multiscale three-dimensional convolutional neural network (3D CNN) architecture that can classify gliomas into low-grade and high-grade gliomas using T1 Gado MRI sequences [20]. In addition, there is still a large room for progress in deep learning network research on survival prediction [22,23]. For example, Xue Feng et al. developed a deep learning model based on 3D U-Net to perform tumor segmentation and extract radiomic features and other clinical features to predict the overall survival of patients. However, their model failed to achieve consistent segmentation performance in testing and validating cases for comparison [22]. Likewise, Xue Fu et al. used a two-branch Dense Net based on multichannel features for prediction, achieving up to 94% classification accuracy. However, the choice of loss function settings, the possibility of insufficient network training, and the possibility of a discrepancy between the generated image and the actual image limit its application in clinical settings [23]. It is unclear which strategies may be effective in predicting gliomas. By studying these studies, we are not using radiomic features alone as predictors, given the complexity of the clinical situation. Rather, we select clinical features that are closely related to prognosis and combine the two for survival prediction. In addition, to avoid the shortcomings of a single prediction model, we improve the fit of multiple learning models and select the best performing set of them. We hope that it will provide some basis and experience for the development of the combination of artificial intelligence and medicine to help clinical problem solving. We chose to optimize the 3D U-Net and expect to provide additional clinical interpretation for GBM classification. We hope it will help us to better understand the relationship between glioblastoma and imaging histological features for survival prediction.

Compared to manual segmentation, which takes a long time, and many professionals, the automatic brain tumor segmentation method of 3D U-net used in our study can save significant time. It can also avoid the possible operational differences between segmentation personnel and reduce errors during the experimental process [10]. Furthermore, pyradiomics can extract our radiomic features of interest following automatic segmentation by deep learning networks, providing quantitative image-based prognostic biomarkers that outperform semi-quantitative estimates [22,24,25]. Axis I3, like f1, is strongly associated with overall survival in the current study and may be a potential prognostic marker. In addition, Yizhou Wan et al. proposed some volumetric features related to glioblastoma recognition and survival using deep learning networks. Tumors with higher proportions of necrosis and contrast enhancement have lower survival rates, while tumors with nonenhanced proportions have improved survival rates [24,28,29].

III.METHODS

We train a deep-learning network (3D U-net) to segment tumors automatically. After the tumor image is segmented, the image features of the region of interest are stored in a deep-learning network. Similarly, after obtaining the image features, machine learning is used to train and validate the data. Figure 1 depicts the flow of the prediction model that we designed.

Fig. 1. Brain tumor classification and prediction flowchart.

IV.EXPERIMENTS AND RESULTS

V.DISCUSSION AND CONCLUSION

In this study, we proposed a fast and automated 3D U-NET framework to segment gliomas, using pyradiomics to extract and filter out important radiomic features. In addition, nine machine learning models were used in conjunction with other clinical features to predict the survival prognosis of glioma patients based on pretreatment MRI. The results of the experiments showed that all nine well-known learning model classification architectures could achieve accurate classification and reliable prediction results. Among them, XGBoost had the best overall performance. In addition, this study architecture is based on 3D neural networks with the potential to merge local and global contextual information through small convolutional kernels. We also used data preprocessing techniques such as intensity normalization and adaptive contrast enhancement, which significantly reduced data heterogeneity and positively affected tumor prediction.

The BraTS challenge has been running since 2012, and there are numerous publications based on participation in this challenge. We compared our segmentation results (e.g., Dice coefficient, Hausdorff distance) with those reported in recent BraTS challenge publications. For example, our 3D U-Net-based segmentation model achieved a Dice coefficient of 0.91 for the enhancing tumor region, which is comparable to the top-performing methods in BraTS2019 (e.g., Isensee et al., 2021, with a Dice score of 0.90). We also highlighted that our method performs particularly well in segmenting necrotic and edema regions, which are often challenging due to their heterogeneous appearance in MRI scans. We also compared our survival prediction results with those from previous studies. For instance, our hybrid model combining radiomic and clinical features achieved a C-index of 0.75, which is competitive with the results reported by Feng et al. (2020) and Lao et al. (2017), who achieved C-indices of 0.71 and 0.73, respectively. We also noted that our model outperforms traditional radiomics-based approaches by incorporating deep learning features, which capture more nuanced patterns in the imaging data.

In addition, there is still room for improvement in our research. There is a lack of opportunity for validation on different external datasets due to the dataset’s small sample content. Moreover, it may not be possible to fully validate all the problems that would occur, resulting in some errors. For example, although all nine models are calibrated in both the training and validation sets, the insufficient amount of data may result in over-fitting. In future research, we will continue to extend the data and test the performance of the learning model. The learning model’s predictions tend to overestimate the risk, so we must find additional ways to optimize the model.

Moreover, while we have identified some predictive radiomic and clinical features, it is unknown whether other features are closely related. Further radiomic and clinical screening is required in future studies. Given that different features may affect overall survival, we used a Pearson correlation coefficient heat map to analyze feature interrelationships and a hybrid model for prediction. However, unaccounted-for associations between radiomic and clinical features may still exist, resulting in biased prediction results. In subsequent studies, we will further explore the relationship between features.