Prediction of the mean transit time using machine learning models based on radiomics features from digital subtraction angiography in moyamoya disease or moyamoya syndrome—a development and validation model study

Introduction

Background

Moyamoya disease (MMD), diagnosed based on cerebral angiography, is a cerebrovascular disease with unknown cause, characterized by chronic progressive stenosis and occlusion of the bilateral internal carotid arteries followed by abnormal proliferation of small vessels at the base of the skull, forming an abnormal vascular network resembling smoke-like changes on angiography (1,2). These fragile, smoke-like vessels provide insufficient cerebral perfusion, causing various ischemic or hemorrhagic symptoms in both children and adults with MMD (3).

Historically, medical imaging has been a qualitative or semi-quantitative modality. With the rapid advances in computer science and the increasing assimilation of medicine and computers, medical data is becoming more available in digital format (4,5). Meanwhile, advances in computing hardware and machine learning algorithms have enabled the development of radiomics. Radiomics is a method to extract undiscovered imaging features into higher dimensional data, which are not accessible by conventional visual image analysis (6). The data might aid in enhancing physicians’ clinical decision-making using advanced machine learning analysis techniques.

Rationale and knowledge gap

Radiomics has proven valuable in the study of pathological lesions. Its concept was initially widely applied mainly in the field of oncology (7). With the progression of research, radiomics has been used in the study of cerebrovascular diseases, such as for predicting the risk of intracranial aneurysm rupture from brain computed tomography angiography (CTA) data (8), and magnetic resonance (MR)-based radiomics to analyze the prognosis of acute cerebral ischemic diseases (9). A predictive feature of radiomics based on precontrast CT imaging could reflect the difference of CTP for severe stenosis or occlusion of the middle cerebral artery (MCA) (10). Li et al. discovered that a computed tomography perfusion (CTP)-based delta-radiomics model has the potential to identify collateral vessel formation after the operation of MMD (11).

Currently, digital subtraction angiography (DSA) remains the gold standard for diagnosing MMD. Concurrently, CTP is a common tool used to assess cerebral hemodynamic status and has been widely used in clinical practice (12,13). CTP has improved patient selection for safe and effective treatment, especially for the evaluation of bypass surgery in patients with MMD.

Objective

Some researchers have tentatively explored the relationship between DSA and CTP in patients with MMD (12). In this study, we aimed to explore the relationship between DSA imaging and mean transit time (MTT; outcome of CTP) from the perspective of radiomics. For MTT, we used DSA images combined with radiomics to create a predictive model and validate the model. We present this article in accordance with the TRIPOD reporting checklist (available at https://cdt.amegroups.com/article/view/10.21037/cdt-23-151/rc).

Methods

Patients

We retrospectively reviewed patients diagnosed with MMD or moyamoya syndrome (MMS) in the Guangdong Provincial People’s Hospital between January 2018 and December 2020. The Institutional Review Board of Guangdong Provincial People’s Hospital approved this research (No. KY-Q-2022-344-02), and the requirement for written informed consent was waived. This study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). A total of 50 patients were diagnosed with MMD or MMS by two experienced physicians (a neurovascular surgeon with 10 years’ experience and a neuroradiologist with 20 years’ experience) according to the diagnostic guidelines for MMD (14).

The patient inclusion criteria were as follows: (I) 18–70 years old; (II) no stroke having occurred within 1 month before admission; (III) diagnosis of MMD or MMS by CTA, MR angiography, or DSA.

The patient exclusion criteria were as follows: (I) comorbidity of other intracranial diseases (intracranial tumors, hydrocephalus, etc.), other intracranial vascular diseases (intracranial aneurysms, intracranial arteriovenous malformations, intracranial arteriovenous fistula, posterior circulation vascular stenosis, etc.), or cervical vascular stenosis (stenosis at the beginning of internal carotid artery and vertebral artery; (II) pregnancy; (III) having not undergone DSA.

The enrolled healthy controls met the following criteria: (I) 18–70 years old; (II) without any intracranial diseases confirmed by neuroimaging. The demographic and clinical data of the patients were extracted from the hospital information system of our center. The research process of this study is detailed in the flow diagram (Figure S1).

DSA images

DSA was performed with Artis Q Zeego (Siemens Healthineers, Erlangen, Germany), and two phases of DSA images were collected: (I) arterial phase of lateral view; (II) capillary phase of lateral view. We collected DSA images of both the left and right sides of healthy controls, and the lesion side only of patients with MMD or MMS. All images were digitally subtracted to remove the skull. An optimal slice of DSA images of the cases was selected for further analysis.

Radiomics features extraction

Two neurosurgeons with more than 10 years’ experience independently segmented the region of interest (ROI) on DSA images using open-source software 3D slicer (version 4.10.20; https://www.slicer.org/). The ROI results were reviewed and censored by a neuroradiologist with 10 years of experience. Discrepancies were settled by consensus discussion. Representative DSA images of lateral arterial phase and capillary phase with corresponding ROIs are shown in Figure S2 and Figure S3. A total of 93 radiomics features, including First Order Statistics, Gray Level Co-occurrence Matrix, Gray Level Size Zone Matrix, Gray Level Run Length Matrix, Neighboring Gray Tone Difference Matrix, and Gray Level Dependence Matrix, were extracted from the ROI of each image using PyRadiomics 2.2.0 (15), Numpy1.13.1, SimpleITK 1.1.0, PyWavelet 1.0.0, and Python 2.7.13.

Data acquisition of cerebral CTP

Cerebral CTP was performed with 128-sprial CT scanner (Philips Healthcare, Amsterdam, Netherlands). Post-processing of images was conducted through Philips IntelliSpace Portal software. We selected L, M4, M6, and M2 regions (corresponding to basal ganglia/thalamus, frontal, parietal, and temporal lobes, respectively) (Figure S4) according to the Alberta Stroke Programme Early CT (ASPECT) score (16). ROIs were segmented by two neuroradiologists independently. Cerebral blood flow (CBF), cerebral blood volume (CBV), time to peak (TTP), and MTT are commonly used parameters in CTP. Cremers et al. (17) showed that MTT (5–6.5 s) has good specificity and sensitivity for cerebral ischemia. Therefore, MTT was selected for further analysis. With additional reference to the previous literature (12,17-19), we considered MTT ≥6 seconds as poor cerebral perfusion status, and vice versa.

Preprocessing and selection of radiomics features

In order to avoid the disaster of dimensionality caused by miscellaneous radiomics features, which may lead to overfitting of the prediction model and reduction of its generalization ability, we sequentially applied a series of feature engineering methods to reduce and screen the original radiomics features in our research, all of which were carried out by R (version 4.1.2, 20211101; R Foundation for Statistical Computing, Vienna, Austria) software. Inter-observer reproducibility was first assessed using intraclass correlation coefficients (ICCs) to retain stable features (20,21). Another junior neurosurgeon with 3 years’ experience, who was blinded to the clinical information, randomly selected 15 cases and conducted image segmentation and feature extraction abiding the same protocol. These data were merely used to calculate ICC and evaluate interobserver concordance. Features with ICC <0.9 were excluded. Ahead of formal feature selection, we performed some preprocessing procedures to standardize radiomics features. Z normalization was applied to unify the magnitude of different features, and therefore improve the comparability and repeatability of the data (22). Missing values were imputed using the K-nearest neighbors algorithm, which is a widely used imputation technique that shows its robustness against parameter tuning and varying numbers of missing values. Then, we implemented Box-Cox transform to solve data skewness and eliminate near-zero variance features, that is, features with low frequencies of a few unique values (23,24). After that, we calculated the Spearman correlation of features. Features with Spearman correlation >0.9 were considered highly correlated and subsequently excluded. After the preprocessing steps, cases were randomly divided into a training set and test set according to the perfusion status at a ratio of 8:2. The least absolute shrinkage and selection operator (LASSO) regression algorithm was then used to further filter and select the radiomics features (25). The LASSO regression algorithm is a famous method that has been widely used to handle high-dimensional data in previous radiomics studies. By setting the parameter λ, the algorithm tends to shrink coefficients of features towards 0, which would make coefficients of redundant features become 0. The optimal λ value was selected according to the minimum error of 10 times cross-verification. Radiomics features with non-zero coefficients at the optimal λ value were chosen to construct the final radiomics signature. At last, we performed single-factor ROC analysis to explore the diagnostic ability of the radiomics signature. Feature selection and dimensionality reduction were carried out on the training set.

Performance evaluation of the prediction model

Two widely used supervised machine learning methods, support vector machine (SVM) and random forest (RF), were exploited to build classic and robust prediction models using a radiomics signature based on the training set. Model parameters were adjusted through 5 times repeated 10-fold cross-validation process on the training set. The performance of models was evaluated on the test set. Corresponding ROC curves were plotted and area under the curve (AUC) values were utilized as evaluation criteria. Youden indexes were also calculated and selected as threshold values, which were used to further calculate models’ confusion matrixes and performance metrics, such as accuracy, sensitivity, specificity, positive predictive value, and negative predictive value. Besides, waterfall plots were created to visualize models’ confusion matrixes on the training set and test set, respectively. Lift curves were also introduced in this study to assess the model’s ability to discover events in a binary classes data set. Brier score is calculated to comprehensively assess the predictive ability of the models. In order to further investigate the relationship between the expected value of model prediction and radiomics signature, we analyzed ceteris-paribus (CP) profiles and partial-dependence (PD) profiles of prediction model. PD was first introduced by Friedman et al. in 2000 and has become increasingly popular in recent years (26).

Statistical analysis

The chi-square test and the Mann-Whitney test were performed to examine the differences between two groups for categoric and continuous variables, respectively. A P value <0.05 was considered statistically significant. All statistical analyses were implemented using R language software (version 4.1.2, 20211101) with the psych, glmnet, caret, tidyverse, pROC, and DALEX packages.

Results

Cohort demographic

A total of 68 cases, including 50 patients with MMD or MMS and 18 healthy controls, were enrolled in the final analysis, the demographic and clinical characteristics of the study population was shown in Table 1. A total of 98 DSA images were collected. Among them, there were 37 cases with bilateral DSA images, 10 cases with right-sided DSA images, and 14 cases with left-sided DSA images. As for perfusion status, high perfusion of the frontal lobe, temporal lobe, parietal lobe and basal ganglia/thalamus was detected in 28, 32, 20, and 51 cases, respectively. Low perfusion of the frontal lobe, temporal lobe, parietal lobe, and basal ganglia/thalamus was detected in 70, 66, 78, and 47 cases, respectively. The training sets and test sets included 79 and 19 cases, respectively.

Features selection and dimensionality reduction

We extracted 93 radiomics features from each image. A total of 38 features were remaining after preprocessing procedures. LASSO regression was applied to further filter features (Figure 1). For the frontal lobe, temporal lobe, parietal lobe, and basal ganglia/thalamus, 4, 3, 3, and 3 features were finally selected and used to construct a corresponding radiomics signature.

Figure 1 LASSO regression for feature selection, each row represents a different subgroup. (B,D,F,H) LASSO coefficient profiles for radiomics features from the frontal lobe (B), temporal lobe (D), parietal lobe (F), and basal ganglia/thalamus (H). Each line represents coefficients of a radiomics feature. (A,C,E,G) 10-fold cross-validation was performed to select tuning parameter (λ) by deviance for the frontal lobe (A), temporal lobe (C), parietal lobe (E), and basal ganglia/thalamus (G). LASSO, least absolute shrinkage and selection operator.

Single-factor ROC analysis

In order to investigate diagnostic ability of selected radiomics features, we performed single-factor ROC analysis on radiomics signatures from the frontal lobe, temporal lobe, parietal lobe, and basal ganglia/thalamus on the whole dataset (Figure 2). The result showed that the AUC values of selected features in all subgroups were greater than 0.7. For the frontal lobe, the AUCs of selected features were all above 0.9 (Figure 2A), and for the temporal lobe and parietal lobe, the AUCs were all greater than 0.8 (Figure 2B,2C). Single-factor ROC analysis demonstrated good diagnostic ability of selected radiomics features.

Figure 2 Single-factor ROC analysis of radiomics signature from frontal lobe (A), temporal lobe (B), parietal lobe (C) and basal ganglia/thalamus (D) on the whole dataset. AUC, area under the ROC curve; ROC, receiver operating characteristic.

Model establishment and evaluation

SVM and RF algorithm were utilized to establish prediction models based on the radiomics signature of the frontal lobe, temporal lobe, parietal lobe, and basal ganglia/thalamus. We plotted ROC curves and calculated corresponding AUC values and Youden indexes (Figure 3). AUC values were chosen as evaluation criteria to select the best prediction models. As for machine learning models, RF models had the best performance on the frontal lobe, parietal lobe, and basal ganglia/thalamus, whereas the SVM model performed the best on the temporal lobe. The AUC values of all final prediction models on the test set were greater than 0.9. For the frontal lobe and parietal lobe, RF models could perfectly distinguish all cases no matter on the training set or test set with AUC values of 1.000 [95% confidence interval (CI): 1.000–1.000], whereas for the temporal lobe and basal ganglia/thalamus, prediction models also had excellent prediction power on the test set with AUC values of 0.962 (95% CI: 0.876–1.000) and 0.922 (95% CI: 0.797–1.000), respectively. In addition, the final prediction models were all built based on positive images of DSA. Arterial phase images were used for building prediction models on frontal lobe, parietal lobe, and temporal lobe, and capillary phase images for the basal ganglia/thalamus. We then chose the Youden indexes of models as threshold values to calculate corresponding confusion matrixes and performance metrics of models (Table 2). The final prediction models achieved a sensitivity of 100.0% and a specificity of 100.0% on the frontal lobe and parietal lobe, respectively, and a sensitivity of 100.0%, 88.9% and specificity of 83.3%, 90.0% on the temporal lobe and basal ganglia/thalamus, respectively. In order to visualize the models’ performance and confusion matrixes, we created waterfall plots on the training set and test set in all subgroups (Figure 4). Lift curves, a tool to examine the number of samples detected by a model above a completely random selection of samples, were also created to assess the ability of a model to detect events in a data set with two classes (Figure 5). The Brier scores of the models are all below 0.25, which show high accuracy of probabilistic predictions of the models (Table 3). Our results described above showed that the final prediction models we built could nicely distinguish cases with high perfusion from those with low perfusion of different brain lobes on the data set (Tables S1-S5).

Figure 3 ROC curves for predictive models with Youden index and corresponding specificity, sensitivity labelled on ROC curve. (A,B) RF model of frontal lobe on the training set (A) and test set (B). (C) SVM model of temporal lobe on training set and test set. (D) RF model of ganglia/thalamus on training set and test set. (E,F) RF model of parietal lobe on the training set (E) and test set (F). AUC, area under the ROC curve; ROC, receiver operating characteristic; RF, random forest; SVM, support vector machine.

Figure 4 Waterfall plots for training set and test set. (A,B) RF model of frontal lobe on training set (A) and test set (B). (C,D) SVM model of temporal lobe on training set (C) and test set (D). (E,F) RF model of parietal lobe on training set (E) and test set (F). (G,H) RF model of basal ganglia/thalamus on training set (G) and test set (H). Threshold values were selected as baseline of x-axis. Cases with prediction values over threshold were classified as high MTT (upward bars), vice versa. True classes of cases were labelled as red (high MTT) or green (low MTT). RF, random forest; SVM, support vector machine; MTT, mean transit time.

Figure 5 The lift charts on test set demonstrated prediction models of subgroups. (A) Frontal lobe, (B) temporal lobe, (C) parietal lobe, (D) basal ganglia/thalamus. Each line shows the percentage of samples found when given percent of samples tested.

CP profiles and PD profiles analysis of prediction models

We conducted CP profiles and PD profiles analysis of the final prediction models and plotted corresponding diagrams (Figure 6). This analysis was performed by using all observations from the dataset for all subgroups. Grey lines were used to display CP profiles that represent each observation in data set. Blue lines on the left-side panel displayed PD profiles that were estimated by the mean of the CP profiles, whereas red lines and green lines on the right-side panel displayed PD profiles representing two classes. From the diagrams, we could see that PD profiles were parallel to most of corresponding CP profiles and had basically the same shape, which means that the selected features are mainly independent and have little interaction with each other. Besides, the PD profiles of two classes are clearly separated, which implies that the selected feature could differentiate two classes nicely. Our PD profiles analysis results show that the final prediction models built has good robustness and consistency.

Figure 6 Partial-dependence profiles for prediction models of subgroups. (A,B) Frontal lobe, (C,D) temporal lobe, (E,F) parietal lobe, (G,H) basal ganglia/thalamus. Grey lines are ceteris-paribus profiles, blue line is partial-dependence profiles for the whole dataset. Red line and green line are partial-dependence profiles that represent 2 classes. The X-axis represents feature value.

Discussion

Key findings

In this study, we used 3D slicer software to extract radiomics features from the 2D DSA images of patients with MMD or MMS and obtained quantitative imaging parameters. These data were then analyzed with the patient’s CTP results and found to be good predictors of the outcome of CTP in patients with MMD or MMS.

Strengths and limitations

The use of radiomics analysis provided a quantitative and objective approach to assessing cerebrovascular disease, specifically MMD and MMS. The study utilized a software tool, 3D slicer, to extract radiomics features from 2D DSA images, which allowed for a comprehensive analysis of the images. The findings of this study have the potential to improve the diagnosis and prediction of outcomes in patients with MMD or MMS. However, the study had some limitations, such as selection bias due to the retrospective nature of the study and a relatively small sample size. A prospective multi-center cohort study with a larger sample size is needed to confirm the results of this study.

In this study, we selected a healthy control group. Our primary concern was that patients who were being considered for alternative diagnoses might produce unreliable results due to potential intracranial ischemia. This could have significant implications for the interpretation of our findings. Therefore, we decided to include a healthy control group to ensure the credibility of our results. We can better understand the characteristics of the patient population and confirm if the differences we observed are disease-related. The purpose of doing so is to provide more comprehensive and accurate research findings, which can serve as more reliable evidence for further clinical applications. In this way, we can better understand the contributions of different factors and provide more effective strategies.

Comparison with similar research

DSA has long been the gold standard in the diagnosis of cerebrovascular disease. For the diagnosis of MMD, DSA results still rely on the subjective judgment of physicians to assess the vascularity of smoke-like changes at the skull base and the staging of MMD, but the information of DSA images obtained by the naked eye alone are still limited. The physician is unable to predict the outcome of the patient’s cerebral perfusion by visualizing and analyzing the anatomical structures in the traditional way.

During the 1980s, researchers started exploring the selection of textural characteristics in medical images for the purpose of identifying clusters of microcalcifications in lung nodules (5,27). With the rapid development of computer science and artificial intelligence, the development of quantitative feature mining and extraction of medical images has been further promoted, and medical images have been transformed into quantitative data for analysis, thus radiomics was born. The research of radiomics has been fruitful in the field of oncology, such as lung cancer (28), liver cancer (29), and glioma (30,31). In the area of cerebrovascular diseases, the use of radiomics to analyze cerebral hemorrhage, stroke, and cerebral aneurysm has also been reported in the literature (8,9,32,33), but in the area of MMD, it has not been reported much.

Explanations of findings

Since angiography results are dynamic continuous imaging data, the images in the arterial and capillary phases are distinctly different. We select the static 2D images in the arterial phase and capillary phase for image feature extraction separately. From the results, it can be ascertained whether the images are positive images or negative images, and whether the images are in the arterial phase or capillary phase; they can well predict the results of MTT in CTP results. MTT is the ratio of CBV to CBF, and is the most reproducible CTP parameter in patients with unilateral symptomatic carotid artery stenosis (34). For the various values measured in CTP imaging (e.g., CBF, CBV, and TTP), the MTT has been established as the surrogate parameter for changes in the microvasculature perfusion (35). MTT was found to have a large role in assessing the ischemic status of patients with aneurysmal subarachnoid hemorrhage (35). Therefore, we chose MTT as the primary parameter of CTP.

We selected the ROI of CTP for analysis according to the ASPECT score, such as the frontal, temporal, parietal, and basal ganglia/thalamic regions. Our research suggested that features extracted from the angiography images can predict the CTP results by either RF or SVM, especially with better performance for the frontal and parietal lobes.

This may be because the frontal and parietal lobe sites are in the location of the small distal branches of the anterior or MCA. The proximal MCA is occluded or severely stenosed due to MMD, resulting in insufficient blood supply to the small distal branches of the MCA. In contrast, the blood supply to the temporal lobe and basal ganglia region is compensated to some extent by the proliferating small vessel which is the reason of smoke-like vessel, so the brain tissue in this area is slightly better perfused than that in the parietal and frontal lobes. This might be the reason that prediction models had superior capability in predicting cerebral perfusion of parietal and frontal lobes.

Implications and actions needed

The findings of this study highlight the potential of radiomics analysis in improving the diagnosis and prediction of outcomes in patients with MMD or MMS. Further research is needed to validate the results of this study in a prospective multi-center cohort with a larger sample size. The implementation of radiomics analysis in clinical practice may require the development of standardized protocols and algorithms for image feature extraction and analysis. Radiologists and clinicians should be aware of the potential benefits of radiomics analysis in cerebrovascular diseases and consider incorporating it into their diagnostic and treatment decision-making processes.

Conclusions

Prediction models based on radiomics features extracted from DSA images demonstrate excellent performance in predicting MTT of CTP in patients with MMD or MMS, which may provide help in future clinical practice.

Acknowledgments

Funding: This study was supported by Guangzhou Science and Technology Key Research and Development Program (No. 202206010130), Medical Simulation Education Research Project of China Medical Education Development Center (No. 2021MNZC37), and Science and Technology Program of Guangzhou (No. 202102020650).

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://cdt.amegroups.com/article/view/10.21037/cdt-23-151/rc

Data Sharing Statement: Available at https://cdt.amegroups.com/article/view/10.21037/cdt-23-151/dss

Peer Review File: Available at https://cdt.amegroups.com/article/view/10.21037/cdt-23-151/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://cdt.amegroups.com/article/view/10.21037/cdt-23-151/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was approved by the institutional ethics board of Guangdong Provincial People’s Hospital (No. KY-Q-2022-344-02) and the requirement for written informed consent was waived. This study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Mertens R, Graupera M, Gerhardt H, et al. The Genetic Basis of Moyamoya Disease. Transl Stroke Res 2022;13:25-45. [Crossref] [PubMed]
2. Zhang X, Xiao W, Zhang Q, et al. Progression in Moyamoya Disease: Clinical Features, Neuroimaging Evaluation, and Treatment. Curr Neuropharmacol 2022;20:292-308. [Crossref] [PubMed]
3. Fang YC, Wei LF, Hu CJ, et al. Pathological Circulating Factors in Moyamoya Disease. Int J Mol Sci 2021;22:1696. [Crossref] [PubMed]
4. Mayerhoefer ME, Materka A, Langs G, et al. Introduction to Radiomics. J Nucl Med 2020;61:488-95. [Crossref] [PubMed]
5. Rogers W, Thulasi Seetha S, Refaee TAG, et al. Radiomics: from qualitative to quantitative imaging. Br J Radiol 2020;93:20190948. [Crossref] [PubMed]
6. Lohmann P, Franceschi E, Vollmuth P, et al. Radiomics in neuro-oncological clinical trials. Lancet Digit Health 2022;4:e841-9. [Crossref] [PubMed]
7. Ludwig CG, Lauric A, Malek JA, et al. Performance of Radiomics derived morphological features for prediction of aneurysm rupture status. J Neurointerv Surg 2021;13:755-61. [Crossref] [PubMed]
8. Liu Q, Jiang P, Jiang Y, et al. Prediction of Aneurysm Stability Using a Machine Learning Model Based on PyRadiomics-Derived Morphological Features. Stroke 2019;50:2314-21. [Crossref] [PubMed]
9. Rauseo E, Izquierdo Morcillo C, Raisi-Estabragh Z, et al. New Imaging Signatures of Cardiac Alterations in Ischaemic Heart Disease and Cerebrovascular Disease Using CMR Radiomics. Front Cardiovasc Med 2021;8:716577. [Crossref] [PubMed]
10. Ren Q, An P, Jin K, et al. A Pilot Study of Radiomic Based on Routine CT Reflecting Difference of Cerebral Hemispheric Perfusion. Front Neurosci 2022;16:851720. [Crossref] [PubMed]
11. Li J, Zhang Y, Yin D, et al. CT perfusion-based delta-radiomics models to identify collateral vessel formation after revascularization in patients with moyamoya disease. Front Neurosci 2022;16:974096. [Crossref] [PubMed]
12. Shi Z, Ma G, Zhang D. Haemodynamic analysis of adult patients with moyamoya disease: CT perfusion and DSA gradings. Stroke Vasc Neurol 2021;6:41-7. [Crossref] [PubMed]
13. Xu J, Dai F, Wang B, et al. Predictive Value of CT Perfusion in Hemorrhagic Transformation after Acute Ischemic Stroke: A Systematic Review and Meta-Analysis. Brain Sci 2023;13:156. [Crossref] [PubMed]
14. Research Committee on the Pathology and Treatment of Spontaneous Occlusion of the Circle of Willis. Guidelines for diagnosis and treatment of moyamoya disease (spontaneous occlusion of the circle of Willis). Neurol Med Chir (Tokyo) 2012;52:245-66. [Crossref] [PubMed]
15. van Griethuysen JJM, Fedorov A, Parmar C, et al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res 2017;77:e104-7. [Crossref] [PubMed]
16. Barber PA, Demchuk AM, Zhang J, et al. Validity and reliability of a quantitative computed tomography score in predicting outcome of hyperacute stroke before thrombolytic therapy. ASPECTS Study Group. Alberta Stroke Programme Early CT Score. Lancet 2000;355:1670-4. [Crossref] [PubMed]
17. Cremers CH, van der Schaaf IC, Wensink E, et al. CT perfusion and delayed cerebral ischemia in aneurysmal subarachnoid hemorrhage: a systematic review and meta-analysis. J Cereb Blood Flow Metab 2014;34:200-7. [Crossref] [PubMed]
18. Han Q, Yao F, Zhang Z, et al. Evaluation of Revascularization in Different Suzuki Stages of Ischemic Moyamoya Disease by Whole-Brain CT Perfusion. Front Neurol 2021;12:683224. [Crossref] [PubMed]
19. Guo X, Yuan X, Gao L, et al. Encephaloduroarteriosynangiosis (EDAS) treatment of moyamoya syndrome: evaluation by computed tomography perfusion imaging. Eur Radiol 2021;31:8364-73. [Crossref] [PubMed]
20. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 2016;15:155-63. [Crossref] [PubMed]
21. Perinetti G. StaTips Part IV: Selection, interpretation and reporting of the intraclass correlation coefficient. South European Journal of Orthodontics and Dentofacial Research 2018;5. [Crossref]
22. Chen Y, Chen TW, Wu CQ, et al. Radiomics model of contrast-enhanced computed tomography for predicting the recurrence of acute pancreatitis. Eur Radiol 2019;29:4408-17. [Crossref] [PubMed]
23. Sunil KC, Ramasree RJ. Dimensionality reduction in automated evaluation of descriptive answers through zero variance, near zero variance and non frequent words techniques - a comparison. Conference: 2015 IEEE 9th International Conference on Intelligent Systems and Control (ISCO). doi: 10.1109/ISCO.2015.7282351.10.1109/ISCO.2015.7282351
24. Box GEP, Tidwell PW. Transformation of the Independent Variables. Technometrics 1962;4:531-50. [Crossref]
25. Huang YQ, Liang CH, He L, et al. Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer. J Clin Oncol 2016;34:2157-64. [Crossref] [PubMed]
26. Friedman J. Greedy Function Approximation: A Gradient Boosting Machine. The Annals of Statistics 2000;29:1189-232.
27. Giger ML, Doi K, MacMahon H, et al. Pulmonary nodules: computer-aided detection in digital chest images. Radiographics 1990;10:41-51. [Crossref] [PubMed]
28. Caruso D, Zerunian M, Daffina J, et al. Radiomics and functional imaging in lung cancer: the importance of radiological heterogeneity beyond FDG PET/CT and lung biopsy. Eur J Radiol 2021;142:109874. [Crossref] [PubMed]
29. Wei J, Jiang H, Gu D, et al. Radiomics in liver diseases: Current progress and future opportunities. Liver Int 2020;40:2050-63. [Crossref] [PubMed]
30. Rudie JD, Rauschecker AM, Bryan RN, et al. Emerging Applications of Artificial Intelligence in Neuro- Oncology. Radiology 2019;290:607-18. [Crossref] [PubMed]
31. Li Y, Liu Y, Liang Y, et al. Radiomics can differentiate high-grade glioma from brain metastasis: a systematic review and meta-analysis. Eur Radiol 2022;32:8039-51. [Crossref] [PubMed]
32. Chen Q, Xia T, Zhang M, et al. Radiomics in Stroke Neuroimaging: Techniques, Applications, and Challenges. Aging Dis 2021;12:143-54. [Crossref] [PubMed]
33. Zhang R, Zhu L, Zhu Z, et al. Apparent diffusion coefficient map based radiomics model in identifying the ischemic penumbra in acute ischemic stroke. Ann Palliat Med 2020;9:2684-92. [Crossref] [PubMed]
34. Waaijer A, van der Schaaf IC, Velthuis BK, et al. Reproducibility of quantitative CT brain perfusion measurements in patients with symptomatic unilateral carotid artery stenosis. AJNR Am J Neuroradiol 2007;28:927-32. [PubMed]
35. Hofmann BB, Fischer I, Engel A, et al. MTT Heterogeneity in Perfusion CT Imaging as a Predictor of Outcome after Aneurysmal SAH. AJNR Am J Neuroradiol 2021;42:1387-95. [Crossref] [PubMed]

(English Language Editor: J. Jones)