Use of bio-informatics assessment schema (BIAS) to improve diagnosis and prognosis of myocardial perfusion data: results from the NHLBI-sponsored women’s ischemia syndrome evaluation (WISE)

Introduction

Previously we showed that the initial (i.e., “original”) interpretation of myocardial perfusion imaging (MPI) was improved by modeling the results of one modality, such as magnetic resonance imaging (MRI), against the results of another modality, such as single photon emission computed tomography (SPECT) for diagnosis and prognosis in women with signs and symptoms of ischemia (1). This approach, termed Decisions Informed by Combining Entities (DICE) better identified women likely to experience a major adverse cardiovascular event (MACE) versus the original reading. However, since the DICE formula incorporated the Original MPI reading it was constrained to convert false negatives to true positives, but did not convert false positives to true negatives. Here we introduce a more general approach, which is related to DICE, termed bio-informatics assessment schema (BIAS). In formulating BIAS we hypothesized that for any given perfusion deficit, contrast in the MPI data is variable due to a biasing interaction of each woman’s pathophysiology and the imaging modality. Identification and incorporation of this systematic bias in interpreting MPI data has potential to additionally convert false positives to true negatives compared to the DICE approach.

Methods

Study population

Among the 935 Women’s Ischemia Syndrome Evaluation (WISE) participants undergoing a clinically indicated coronary artery angiography, a sub-population consisting of 213 women with suspected myocardial ischemia were recruited and underwent a clinically indicated gated-SPECT evaluation and a study-directed MRI evaluation. This prospective sub-study was performed at a single WISE site, the University of Alabama at Birmingham, and included those with no contraindications for MR examination. All subjects provided written informed consent using forms and procedures approved by the Institutional Review Board. The MR and gated-SPECT studies were performed on the same day and readers were masked to all other data. Coronary angiography was read at a WISE core laboratory.

Baseline MPI and LV function evaluation

The WISE study design and methodology have been previously described (2,3). In brief, upon enrollment, demographic, CAD risk factors, medical and reproductive history, and functional capacity data were collected as well as blood sampling for Lipid Core Laboratory evaluation. Only patients with complete data from the implementation phase were included for this analysis (n=130).

Gated-SPECT

In brief, the gated-SPECT examination was performed in parallel with the MR examination using dipyridamole (0.56 mg/kg infused over 4 minutes) to induce hyperemia, with methoxyisobutylisonitrile (MIBI) (low dose/high dose) used for both baseline and hyperemic gated-SPECT MPI (4). End-systolic and end-diastolic volumes were extracted with minimal user interaction from a 3D analysis program. Regions of myocardial perfusion deficit were identified by consensus of experienced SPECT readers.

Cardiovascular magnetic resonance

MRI cine images were acquired using a Philips ACS 1.5T scanner (Philips Medical System, Best, The Netherlands). Non-invasive MPI was performed in the short axis orientation using a bolus injection of gadolinium (0.1 mmol/kg at a rate of 4–6 mL/sec) followed by a 10-mLsaline flush. Regions of myocardial perfusion deficit were determined by a consensus of experience MRI readers who then provided the qualitative reading, MRI_QL. A semi-quantitative reading (MRI_SQ) of the MPI defect was performed using the product of the normalized uptake slope and signal gain from baseline to peak (5). Additionally, the myocardial flow reserve (MFR) was calculated as the ratio of the normalized stress to rest uptake slope.

BIAS model

Generation of the BIAS models is described in detail in the Appendix. In brief, a BIAS score is generated by comparing two MPI modalities, effectively yielding a propensity score such that as the BIAS score increases the probability of disease and adverse events increases (Figure 1) while the contrast of disease in the MPI images decreases. For a comparison involving MRI and SPECT, two BIAS scores are generated, one for each modality. As an illustrative example of the manner in which the BIAS score can be used to augment MPI interpretation, two thresholds were retrospectively set (I) a high BIAS score threshold, above which all patients are identified as MPI positive and (II) a lower BIAS threshold, below which all patients are identified as MPI negative, thereby defining a band of BIAS values between the two thresholds in which the Original MPI interpretations were used (details given in the Appendix). This approach was applied to the MRI_SQ, MRI_QL and SPECT data sets to generate a BIAS-augmented MPI interpretation.

Figure 1 The occurrence of MACE (red line) and maximum coronary artery stenosis (blue line) are plotted against tiles of BIAS score by (A) MRI and (B) SPECT. The * indicates that rate increases with BIAS score (P<0.05). MRI, magnetic resonance imaging; BIAS, bio-informatics assessment schema; SPECT, single photon emission computed tomography; MACE, major adverse cardiovascular event.

Follow-up procedures

Follow-up consisted of a scripted telephone interview performed by an experienced WISE Research Coordinator at 6-weeks after enrollment and annually thereafter. The major adverse cardiovascular events (MACE) collected were cardiovascular-related mortality, first incidence of nonfatal myocardial infarction (MI) or hospitalization for heart failure. Follow-up was conducted for 38±14 months. In the event of death, a death certificate and/or hospital record was obtained and a panel of experts adjudicated whether death was cardiovascular related using predetermined criteria.

Statistical analysis

Continuous values were presented as mean ± SD and categorical variables as percent frequency. Continuous clinical and demographic characteristics were compared between groups using the independent samples t-test; the chi-square test was used for categorical comparisons. Analysis of variance (ANOV) was conducted for variables grouped by 8-tiles of BIAS score. Analysis was performed on twelve cardiac regions selected from the standard 17 region model (six regions per short-axis slice for two slices) (6). For ischemia detection, the Original interpretations of the MRI_SQ, MRI_QL and SPECT were used. Sensitivity was defined as the number of successful predictions of MACE divided by the total number of MACE. Specificity was defined as the number of successful predictions of lack of MACE divided by the total number of MACE-free women. Kaplan-Meier plots of time to MACE were performed for patients grouped by quartiles of (I) MRI-BIAS score and (II) SPECT-BIAS score. For the MRI_SQ, MRI_QL and SPECT readings, Kaplan-Meier plots of time to MACE were performed for the Original MPI interpretation, the DICE-augmented interpretation and the BIAS-augmented interpretation. Area under the curve was calculated for ROC analysis for raw BIAS scores and for the MPI designations of perfusion deficit for prediction of MACE and CAD. The population was split into two equal groups, (I) to develop the BIAS equations and (II) to test the BIAS equations (see Appendix for details). All statistical tests were two-tailed and a P value <0.05 was considered to be statistically significant. Statistical analyses were performed using PASW 18.0 (SPSS Inc., Chicago, Illinois, USA).

Results

Population characteristics and data

The mean age of women was 59±12 years (range, 31–86 years); 31% were ethnic minorities, primarily African-Americans. Demographic data for patients are summarized in Table 1. At follow-up, a MACE occurred in 14 women (11%) consisting of 7 cardiovascular deaths, 4 hospitalizations for heart failure, and 3 non-fatal MI’s. CAD was present in 13 (10%) of women. The SPECT-BIAS score negatively correlated with EF (−0.58, P<0.001) as did the MRI-BIAS score (−0.67, P<0.001). Kaplan-Meier analysis for prediction of MACE for quartiles of the MRI-BIAS score (P<0.005) and the SPECT-BIAS score (P<0.001) are shown in Figure 2.

Table 1 Demographics
Full table

Figure 2 Kaplan-Meir plots are shown for MACE for (A) tiles of MRI BIAS score and (B) tiles of SPECT BIAS score with green, blue, red, and purple, for tiles 1, 2, 3, and 4, respectively. MACE, major adverse cardiovascular; BIAS, bio-informatics assessment schema; MRI, magnetic resonance imaging; SPECT, single photon emission computed tomography.

BIAS-augmented MPI interpretation

The AUC for ROC analysis of prediction of MACE for the two BIAS scores were 0.81 and 0.92 for MRI and SPECT, respectively. Based on this and the observed distribution of CAD and MACE with BIAS score (Figure 1) we produced a BIAS-augmented interpretation of the MPI data (see Appendix for details). Table 2 shows the AUC for ROC analysis of the Original, DICE, and BIAS-augmented interpretations, in which is can be seen that for MACE prediction, AUC progressively increases from the Original (0.60±0.06), to DICE (0.74±0.05) and BIAS (0.80±0.02) (P<0.01). For the illustrative examples of the retrospectively defined BIAS-augmented interpretations, the sensitivity for prediction of MACE dramatically increased from the Original interpretation (74±5 vs. 42±7, P<0.05) while the specificity did not significantly change (83±1 vs. 82±3). For comparison, Kaplan-Meier plots of the Original, DICE-augmented, and BIAS-augmented MPI interpretations of MRI_SQ, MRI_QL and SPECT are shown in Figures 3-5, with P values indicated in each plot showing a progression of increasing significance.

Table 2 Sensitivity and Specificity for CAD and MACE
Full table

Figure 3 Kaplan-Meir plots are shown for MACE for (A) the Original MRI_SQ interpretation of the MPI data; (B) the previously described DICE-augmented MRI_SQ interpretation and (C) the BIAS-augmented MRI_SQ interpretation. Where MRI_SQ is the semi-quantitative MRI interpretation of the MPI data. MACE, major adverse cardiovascular; DICE, decisions informed by combining entities; BIAS, bio-informatics assessment schema; MRI_SQ, MRI semi-quantitative perfusion assessment.

Figure 4 Kaplan-Meir plots are shown for MACE for (A) the Original MRI_QL interpretation of the MPI data; (B) the previously described DICE-augmented MRI_QL interpretation and (C) the BIAS-augmented MRI_QL interpretation. Where MRI_QL is the qualitative MRI interpretation of the MPI data. MACE, major adverse cardiovascular; DICE, decisions informed by combining entities; BIAS, bio-informatics assessment schema; MRI_QL, MRI qualitative perfusion assessment.

Figure 5 Kaplan-Meir plots are shown for MACE for (A) the original SPECT interpretation of the MPI data; (B) the previously described DICE-augmented SPECT interpretation and (C) the BIAS-augmented SPECT interpretation. Where SPECT is the single photon emission computed tomography interpretation of the MPI data.

Discussion

We hypothesized that the manifestation of ischemic heart disease in MPI is dependent on an interaction between the woman’s pathophysiology and imaging modality. By comparing two different MPI modalities we identified parameters relating to heart chamber volume and myocardial wall thickness that progressively indicated the presence of disease and MACE. We noted that the generally low sensitivity to MACE in the Original MPI interpretations was largely due to under identification of disease in patients with higher BIAS scores, indicating that disease clarity in MPI is progressively weakened even as the presence of disease progressively increases. Further, the Original MPI interpretation tended to over call disease at low BIAS scores, which doubly hindered MPI interpretation. We demonstrated that even when used in isolation of the MPI data, the BIAS scores dramatically increased the AUC in ROC analysis compared to the Original and DICE-augmented interpretations.

BIAS—augmented interpretation

As an illustrative example, we showed how the Original MPI analysis could be augmented using the BIAS scores. Here, the high concentration of CAD severity and MACE in women with a high BIAS score (both by MRI and SPECT) suggested that interpretation of MPI data could be omitted, and in our example patients were declared to be MPI positive above a certain threshold. Further, at low BIAS scores, the presence of disease and the occurrence of MACE was so low that, again, interpretation of MPI data was omitted, and patients were declared to be MPI negative. Thus, for both MRI and SPECT, the Original MPI interpretations were only used for a narrow band of BIAS scores (0.17 to 0.23 for MRI and 0.14 to 0.18 for SPECT). For these patients, it can be seen from the BIAS-augmented Kaplan-Meier curves of Figures 3,4 that the semi-quantitative and qualitative interpretations of the MRI data produce the same results. That is, for the narrow band of BIAS scores indicated the interpretation of MPI data by two different criteria were in perfect agreement. One interpretation of this is that at intermediate BIAS scores the true presence of disease is clear, but as the BIAS score increases the clarity of image-based evidence of disease progressively decreases, leading to wide-spread disagreement between different MPI interpretation criteria. The physiologic phenomena that governs this behavior is at present unknown, but may be related to the increased amount of myocardium and decreasing function associated with increasing BIAS scores. This is uniquely distinct from the progressive influence of co-morbidities, which influence the occurrence of MACE but are not postulated to influence the clarity of disease in the MPI images.

As indicated in Table 2, the agreement between the BIAS-augmented MPI interpretation and MACE was generally better than the agreement with CAD using cath lab data. Others have shown that decreased MFR, even in the absence of flow-limiting coronary artery disease may be associated with microvascular disease (7). Thus, consideration of these issues indicates that interpretation of coronary artery stenoses without consideration of myocardial perfusional conditions may lead to an inadequate risk assessment.

MPI measures of ischemia

The Original MRI_SQ identification of ischemia involved a combination of the slope and peak signal reached for the stress uptake, with a threshold based on the MFR level. Others have used just one of these indices while yet others have derived quantitative MPI values from advanced analyses of the MPI time-intensity curves (8,9). Similarly, SPECT has its own set of quantitative criteria (10). Here we showed that when using the BIAS score to augment MPI interpretation (Figures 3,4,5) it is apparent that the BIAS scores dominated, with specific MPI data interpretation only marginally contributing. In the examples given here of incorporating the MPI interpretation with the BIAS score, the AUC of the ROC analysis for the MPI augmented interpretations never exceeded the AUC of the raw BIAS scores. It seems likely that separate MPI interpretation criteria or thresholds of significance will be needed to be applied based on the BIAS score. Determination of this will likely require further research with a larger patient population.

Diagnostic versus prognostic

Multiple factors such as MFR and processing noise are known to affect the reproducibility, diagnostic and prognostic accuracy of MPI assessed by MRI (5,11-13). Further, assessing MPI success using the degree of agreement with the cath lab-determined coronary artery stenosis level is not straight forward due to MRI’s sensitivity to microvascular disease, collateral circulation and myocardial flow reserve (14). Thus, complete agreement with the cath lab is not generally expected, and may not even be an ideal to strive for (otherwise there would be no complementary value to the two data sets) (15). Similar issues have guided the SPECT community to transition from using the test for diagnostic to prognostic purposes (16). Further work is required to use the BIAS-augmented reading to separately improve diagnostic and prognostic interpretations.

Limitations

While the BIAS score was constructed without knowledge of CAD status, MACE, and other cardiovascular risk scores, the thresholds for the BIAS-augmented interpretation were set based on knowledge of outcomes, and as such can only be considered to be a demonstration of the manner in which BIAS could be used. However, no re-reading of images was performed, and analysis was performed on data that was extracted without knowledge of BIAS scores or outcomes. Analysis was restricted to post pilot data due to the uniformity of the MRI_SQ data set, limiting the number of patients available for analysis. Data were only obtained from one site. Future research needs to focus on developing a prospective study to validate these findings in different cohorts and determine how they differ from men.

Conclusions

In women with suspected myocardial ischemia, BIAS scores were generated by modeling measurements between SPECT and MRI MPI to determine the factors that influence a positive study. Increasing values of the BIAS score by both MRI and SPECT corresponded to the increasing prevalence of CAD and MACE. Further, increasing values of the BIAS score corresponded to lower image-based evidence of disease. For the MRI and SPECT data a band of BIAS values was identified in which conventional MPI interpretation was applied, with higher BIAS values indicating positive interpretation and lower BIAS values indicating negative interpretation. The BIAS-augmented adjudication of ischemia better predicted MACE than the Original readings.

Based on the hypothesis that MPI contrast is dependent on variables other than myocardial perfusion level, logistic regression analysis was performed to model the MPI reading of a target modality (e.g., gated-SPECT) by entering into the model variables such as ventricular volumetric data derived from a second modality (e.g., MRI) but importantly explicitly omitting MPI status assessed by the second modality. Due to the relatively small patient population and the relatively low event rate it was not possible to totally randomly split the patients into two groups for purposes of (I) developing the BIAS scores and (II) assessing the BIAS scores. However, the approach that we adopted was to split the population into two random groups (65 per group) with the exception that the assessment group contained all the patients with MACE. Further, to establish which parameters had sufficient statistical significance for inclusion in the BIAS models we determined this using the full data set. However, to generate the BIAS models, the parameters were only derived from the development group (i.e., the group with no incidence of MACE). The logistic regression equations (LRE) thus generated were converted into probabilities by the standard formula

Probability = e^LRE/ (1+e^LRE) [1]

This probability function represents the physiologic bias that influences the prominence of perfusion contrast in MPI. Here, two regression models were constructed using the MRI and SPECT data sets:

MRI

The logistic regression equation predicting the Original SPECT-identified ischemia using MRI data is:

SPECT_model =−5.258 + ESVi × 0.062 + Wall × 0.212 [2]

Where ESVi (mL/m²) is end systolic volume index and Wall is the average myocardial wall thickness (mm) at end-diastole measured using MRI.

SPECT

The corresponding logistic regression equation modeling the Original MRI_SQ-identified ischemia using SPECT data is:

MRI_model =−3.535 + EDVi × 0.04 [3]

Where EDVi (mL/m²) is the end diastolic volume index measured by SPECT.

These logistic regression equations (LRE) were converted into the probability of ischemia being present (BIAS score) by use of Eq. [1]. To demonstrate their validity the MRI and SPECT BIAS scores were arbitrarily binned into 8 tiles and entered into an ANOVA evaluation to examine how the maximum coronary artery stenosis level by cath varied with BIAS score, Figure 1. For both MRI and SPECT BIAS scores, maximum stenosis increased with increasing BIAS score (MRI correlation r=0.22, P=0.08 and SPECT correlation r=0.28, P<0.05). Similarly, the MACE rate increased with increasing BIAS score (MRI correlation r=0.415, P<0.001 and SPECT correlation r =0.55, P<0.001).

BIAS-augmented MPI thresholds for MRI

Based on knowledge of outcomes, progressive thresholds for use of MRI_SQ and MRI_QL data were set based on the 8-tiles boundaries of the MRI BIAS score:

Ischemia is positive IF BIAS score >0.23

OR IF (0.23> BIAS score >0.17) & Original MRI MPI is positive [4]

Where “IF”, “OR” and “&” are logical operations.

BIAS-augmented MPI thresholds for SPECT

Based on knowledge of outcomes, progressive thresholds for use of SPEC data were set based on the 8-tiles boundaries of the SPECT BIAS score:

Ischemia is positive IF BIAS score >0.18

OR IF (0.18> BIAS score >0.14) & Original SPECT MPI is positive [5]

Where “IF”, “OR” and “&” are logical operations.

Acknowledgements

We are grateful to Bracco, Princeton, NJ, for providing the ProHance contrast agent.

Funding: This work was supported by contracts from the National Heart, Lung and Blood Institutes, nos. N01-HV-68161, N01-HV-68162, N01-HV-68163, N01-HV-68164, RO1-HL-073412-01, grants U0164829,U01 HL649141, U01 HL649241, and grants from the Gustavus and Louis Pfeiffer Research Foundation, Danville, New Jersey, The Women’s Guild of Cedars-Sinai Medical Center, Los Angeles, California, The Ladies Hospital Aid Society of Western Pennsylvania, Pittsburgh, Pennsylvania, and QMED, Inc., Laurence Harbor, New Jersey, and the Edythe L. Broad Endowment, the Barbra Streisand Women’s Cardiovascular Research and Education Program, and the Linda Joy Pollin Women’s Heart Health Program, Cedars-Sinai Medical Center, Los Angeles, California.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: All subjects provided written informed consent using forms and procedures approved by the Institutional Review Board.

References

1. Doyle M, Pohost GM, Merz CN, et al. Improved diagnosis and prognosis using Decisions Informed by Combining Entities (DICE): results from the NHLBI-sponsored Women's Ischemia Syndrome Evaluation (WISE). Cardiovasc Diagn Ther 2013;3:216-27. [PubMed]
2. Doyle M, Weinberg N, Pohost GM, et al. Prognostic value of global MR myocardial perfusion imaging in women with suspected myocardial ischemia and no obstructive coronary disease: results from the NHLBI-sponsored WISE (Women's Ischemia Syndrome Evaluation) study. JACC Cardiovasc Imaging 2010;3:1030-6. [Crossref] [PubMed]
3. Merz CN, Kelsey SF, Pepine CJ, et al. The Women's Ischemia Syndrome Evaluation (WISE) study: protocol design, methodology and feasibility report. J Am Coll Cardiol 1999;33:1453-61. [Crossref] [PubMed]
4. DePuey EG, Parmett S, Ghesani M, et al. Comparison of Tc-99m sestamibi and Tl-201 gated perfusion SPECT. J Nucl Cardiol 1999;6:278-85. [Crossref] [PubMed]
5. Doyle M, Fuisz A, Kortright E, et al. The impact of myocardial flow reserve on the detection of coronary artery disease by perfusion imaging methods: an NHLBI WISE study. J Cardiovasc Magn Reson 2003;5:475-85. [Crossref] [PubMed]
6. Cerqueira MD, Weissman NJ, Dilsizian V, et al. Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart. A statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association. Int J Cardiovasc Imaging 2002;18:539-42. [PubMed]
7. Panting JR, Gatehouse PD, Yang GZ, et al. Abnormal subendocardial perfusion in cardiac syndrome X detected by cardiovascular magnetic resonance imaging. N Engl J Med 2002;346:1948-53. [Crossref] [PubMed]
8. Al-Saadi N, Nagel E, Gross M, et al. Noninvasive detection of myocardial ischemia from perfusion reserve based on cardiovascular magnetic resonance. Circulation 2000;101:1379-83. [Crossref] [PubMed]
9. Hsu LY, Groves DW, Aletras AH, et al. A quantitative pixel-wise measurement of myocardial blood flow by contrast-enhanced first-pass CMR perfusion imaging: microsphere validation in dogs and feasibility study in humans. JACC Cardiovasc Imaging 2012;5:154-66. [Crossref] [PubMed]
10. Bailey DL, Willowson KP. An evidence-based review of quantitative SPECT imaging and potential clinical applications. J Nucl Med 2013;54:83-9. [Crossref] [PubMed]
11. Morton G, Ishida M, Schuster A, et al. Perfusion cardiovascular magnetic resonance: Comparison of an advanced, high-resolution and a standard sequence. J Cardiovasc Magn Reson 2012;14:34. [Crossref] [PubMed]
12. Goykhman P, Mehta PK, Agarwal M, et al. Reproducibility of myocardial perfusion reserve - variations in measurements from post processing using commercially available software. Cardiovasc Diagn Ther 2012;2:268-77. [PubMed]
13. Bratis K, Nagel E. Variability in quantitative cardiac magnetic resonance perfusion analysis. J Thorac Dis 2013;5:357-9. [PubMed]
14. Shufelt CL, Thomson LE, Goykhman P, et al. Cardiac magnetic resonance imaging myocardial perfusion reserve index assessment in women with microvascular coronary dysfunction and reference controls. Cardiovasc Diagn Ther 2013;3:153-60. [PubMed]
15. Hachamovitch R, Nutter B, Hlatky MA, et al. Patient management after noninvasive cardiac imaging results from SPARC (Study of myocardial perfusion and coronary anatomy imaging roles in coronary artery disease). J Am Coll Cardiol 2012;59:462-74. [Crossref] [PubMed]
16. Hachamovitch R, Berman DS, Shaw LJ, et al. Incremental prognostic value of myocardial perfusion single photon emission computed tomography for the prediction of cardiac death: differential stratification for risk of cardiac death and myocardial infarction. Circulation 1998;97:535-43. [Crossref] [PubMed]