Reproducibility of myocardial perfusion reserve - variations in measurements from post processing using commercially available software

Background

First-pass stress perfusion cardiac magnetic resonance imaging (CMRI) can detect stress-induced myocardial hypoperfusion in patients with coronary artery disease (CAD) with a high sensitivity and specificity (1,2). In the setting of obstructive CAD, the visual analysis of first pass perfusion images is performed for detection of segmental hypoperfusion in a vascular territory supplied by a culprit epicardial coronary artery.

Stress perfusion CMRI can also be abnormal in patients with angina who do not have obstructive CAD. A subset of these patients, typically women, have microvascular coronary dysfunction (MCD) which carries an adverse cardiovascular prognosis (3-5). Due to high spatial and temporal resolution, stress CMRI is particularly helpful in detecting diffuse subendocardial and subepicardial hypoperfusion thought to be due to ischemia observed in the setting of MCD (6,7).

The semi-quantitative analysis of perfusion CMRI data utilizes software to measure time intensity curves within myocardium and the left ventricular cavity for computerized calculation of myocardial perfusion reserve index (MPRI) which is the ratio of the adjusted upslope curves for stress compared to rest first pass perfusion. Specialized knowledge of CMRI anatomy and an understanding of the patterns of artifact that cab be seen is needed by imaging physicians, and guidelines for specialized training requirements have been developed (8). User interaction is required for adjustment of software generated endocardial and epicardial borders and the position of the left ventricular cavity region of interest. The user also determines the first and last points of the time intensity curve to be included in the calculation of the upslope. If there is cardiac or respiratory motion during the first pass of contrast, image contours need to be adjusted on a frame by frame basis.

It is important to define accuracy of MPRI measurement so that differences between groups or values obtained in individuals can be defined as being real or potentially due to known variation in measurement. The reproducibility of MPRI has been evaluated in small groups and reported as CoV up to 19% (9). This variation is potentially due to a combination of factors including variation in stress test response, image acquisition/quality, and variation in measurements at the time of post processing.

There are several commercially available software programs for evaluation of stress perfusion CMRI. This study aims to define the inter- and intra- observer variation for measurements made using CAAS MRV 3.3 software (Pie Medical Imaging B.V., Netherlands).

Methods

Data source

The perfusion CMRI data were retrospectively analyzed from twenty adenosine stress first pass perfusion studies performed in 18 women who were participating in an IRB approved clinical trial under the care of the Women’s Heart Center, Cedars-Sinai Heart Institute, Los Angeles, California. This trial was exclusively for women with signs and symptoms of ischemia and no obstructive coronary atherosclerosis (epicardial coronary stenosis <50% luminal diameter stenosis) on left heart catheterization. Those with obstructive coronary atherosclerosis (>50% CAD) were excluded from this trial. Microvascular and endothelial coronary dysfunction was suspected in these symptomatic women with open coronary arteries, and some underwent invasive coronary reactivity testing as part of clinical care to definitively diagnose microvascular coronary dysfunction (MCD) based on a previously published protocol (10).

Ten CMRI scans were performed in eight women with signs and symptoms of myocardial ischemia (chest pain, abnormal stress testing) in the absence of obstructive CAD (epicardial coronary stenosis <50% luminal diameter stenosis), in whom microvascular coronary dysfunction (MCD) was suspected. Two women with suspected MCD had repeat scans because they were part of another trial evaluating ranolazine in treatment of MCD (11). Ten CMRI scans were from an asymptomatic group of healthy female volunteers, who were asymptomatic, and self-identified as having no heart disease or heart disease risk factors, and had a negative exercise stress test by the standard Bruce Protocol. All perfusion data was included for post processing.

Cardiac Magnetic Resonance Imaging(CMRI) Protocol

CMRI perfusion imaging was performed as part of a standardized protocol that included anatomic, cine, first pass perfusion and delayed enhancement imaging (12,13).

The CMRI data was acquired in the supine position on a 1.5-Tesla CMRI scanner (Siemens Sonata, Erlangen, Germany) using ECG-gating and a phased array coil (CP Body Array Flex, Siemens Medical Systems, Erlangen, Germany). Vasodilator stress was performed using adenosine (140 mcg/kg/min IV over 4 minutes) and Gadolinium (OptiMARK®) 0.05 mmol/kg was administered at 4 mL/s via a separate IV catheter, followed by 30 mL saline at 4 mL/s, commencing after two minutes of adenosine. Rest perfusion was performed 10 minutes later, with images obtained in matched basal, mid and distal left ventricular short axis slices. Perfusion images were acquired in end-expiration with GRE-Epi hybrid pulse sequence, imaging every heart beat. Standard Siemens Medical Imaging sequence was used for the studies: GRE-Epi perfusion pulse sequence with TE 1.17 msec, TR(slice) 148 msec, Flip angle 20 degrees, bandwidth 1,420 Hz, parallel imaging R=2, echo train length 4, matrix 160 mm × 79 mm, voxel size 2.7 cm × 2.2 cm × 8.0 cm.

CMRI Quantitative Analysis

Quantitative analysis of the first pass perfusion images was performed using CAAS MRV 3.3 software (Pie Medical Imaging B.V., Netherlands). There were three separate post processing episodes. Two independent users (LT and PG) experienced with the software and post processing of stress perfusion CMRI data, evaluated the 20 studies at separate times, without knowledge of clinical data. The analysis of images was repeated three months later by one user (PG). The software interactions for data analysis were carefully standardized between users. Each user performed the following steps:

STEP 1. The placement of contours was performed according to the software manufacturer’s specifications. The contours were manually adjusted, frame by frame if needed, to optimize sampling of the myocardium. The endocardial contours were placed without encroaching on the LV cavity and the papillary muscles were included in the cavity (Figure 1). The left-right ventricular (LRV) junction point was placed at the inferior portion of the interventricular septum.

Figure 1. First pass CMRI perfusion images. Images A and B demonstrate short axis two chamber view through right ventricle (RV) and left ventricle (LV)

STEP 2. The intensity over time curves at rest and stress for an AHA standard 17 segment model were generated by the software for each myocardial region: whole myocardium, endocardial and epicardial half of the myocardium, which represent subendocardial and subepicardial regions, respectively. The apical segment (17^th segment), was not calculated since the first pass perfusion images are acquired in short axis and do not include the apex. These curves were inspected, with re-adjustment of the contours by the user if needed, in order to optimize the segmental time intensity curve slopes.

STEP 3. The observers manually defined the starting point (T0 cycle) and the ending point (T end) of time intensity curves. T0 was set at the baseline point immediately prior to the upslope and T end was placed at the point where peak intensities were reached (Figure 2). The ratio of the maximum upslope of the selected curve, which corresponds to the specific myocardial segment, over the maximum upslope of the LV cavity curve, gives the relative upslope (RU). MPRI for each segment is then calculated by RU at stress divided by RU at rest. Sub segment data is also generated by the software for subendocardial and subepicardial MPRI.

Figure 2. Intensity over time curves. Manually placed epicardial (blue) and endocardial (red) contours at rest (A) and stress (B) are used to derive intensity over time curves. The red, green and blue curves correspond to basal, mid, and apical left ventricular cavity intensities. The curves correspond to each myocardial segment based on the 17-segment AHA model. The maximum upslope is calculated between T0 and T end (arrows), systematically assigned to capture the left ventricular input

First pass CMRI perfusion images. Images A and B demonstrate short axis two chamber view through right ventricle (RV) and left ventricle (LV)

Intensity over time curves. Manually placed epicardial (blue) and endocardial (red) contours at rest (A) and stress (B) are used to derive intensity over time curves. The red, green and blue curves correspond to basal, mid, and apical left ventricular cavity intensities. The curves correspond to each myocardial segment based on the 17-segment AHA model. The maximum upslope is calculated between T0 and T end (arrows), systematically assigned to capture the left ventricular input

Statistical analysis

T test was used to compare mean values of continuous variables of normal and abnormal studies and ^χ2 test was used to analyze categorical variables. Two repeatability coefficients (RC), defined as 1.96× standard deviation of the differences between pairs parameters (14,15), were determined to test inter/intra-observer reliability. Variance comparison tests were employed to compare two RC. Pearson statistic was used to calculate intra-class correlation coefficient (ICC). Bootstrap re-sampling method was used to compare ICC. Coefficient of variation (CoV) was also calculated by dividing the standard deviation of the difference by the mean (9). P<0.05 was considered significant. Statistical analysis was performed using Statistical Analysis System (ver. 9.1; The SAS Institute, Cary, NC) and STATA 10.1 (Stata Corp, College Station, TX).

Results

Subject characteristics

Subject characteristics are shown in Table 1 for the whole group and subgroups. There was a high frequency of adenosine-related side effects, but no significant complications occurred and all subjects completed the entire imaging procotol.

Table 1 Baseline demographic variables and risk factors
Full table

Reproducibility of MPRI

The mean MPRI values obtained by the three separate post processing episodes, for the whole group are shown in Table 2, and MPRI data is presented for the whole 16 segment as well as individual slices for both subendocardial and subepicardial regions. The values obtained were similar, as shown in the Bland-Altman analysis for mean and mid-ventricular MPRI for inter-observer and intra-observer calculations (Figures 3,4). The difference was smaller for intra- observer compared to inter-observer measurements.

Table 2 Results of three separate software post processing episodes for calculation of MPRI
Full table

Figure 3. Inter-observer and intra-observer MPRI data for the whole myocardium. Bland Altman plots depict mean differences in MPRI for the whole myocardium for inter- and intra-observer data. A range of agreement was defined as mean bias +/- 2 SD of the mean bias

Figure 4. Inter-observer and intra-observer MPRI data at the mid ventricular level alone. Bland Altman plots depict mean differences in MPRI when considering the mid ventricular slice alone for inter- and intra-observer data. A range of agreement was defined as mean bias ±2 SD of the mean bias

The inter- and intra-observer reproducibility of left ventricular MPRI calculated by ICC and expressed in terms of RC and CoV is shown (Tables 3,4). The inter- and intra-observer variability of MPRI based on subgroups is shown in Table 5. There was a strong inter- and intra-observer ICC within each myocardial region (whole, subendocardial, subepicardial). There was no difference between individual slices in terms of the ICC, RC and CoV. All measures of reproducibility were better for intra- observer rather than inter- observer data.

Table 3 Inter-Observer reproducibility of mean left ventricular MPRI (n=20)
Full table

Table 4 Intra-Observer reproducibility of mean left ventricular MPRI data (n=20)
Full table

Table 5 Inter- and Intra-Observer variability in MPRI comparison of subgroups
Full table

Discussion

Cardiac magnetic resonance imaging (CMRI) is a rapidly evolving tool in the diagnosis of ischemic heart disease, both obstructive coronary artery disease that leads to segmental hypoperfusion and detection by CMRI, but also diffuse hypoperfusion which can be due to microvascular and endothelial coronary dysfunction (7,16) The use of quantitative analysis of CMRI contrast enhanced images has been extensively studied in viability imaging (17). The presence and quantitative extent of delayed enhancement is of prognostic importance for patients with and without obstructive CAD (18,19). It has been reported that measurement of MPRI by CMRI is useful for defining presence of epicardial coronary artery disease in patients with CAD (9,20). However reproducibility of the measurements made by semi quantitative analysis of data using this standard commercially available software has not previously been described. This is particularly important when interpreting results of semi quantitative analysis when defining the presence or absence of disease, when evaluating changes in MPRI with treatment, or when studying differences between groups of subjects in research trials.

The use of semi-quantitative methods to evaluate dynamic myocardial blood flow in positron emission tomography (PET) using three different softwares was shown to be highly correlated for both regional (specific vascular territory) and global myocardial flow. Furthermore, the interobserver variability for resting myocardial blood flow and blood flow reserve was highly correlated (R²=0.91-0.99) for the three methods utilized (21). Slomka et al. (15) have also shown that visual and automated quantitative analysis of gated myocardial perfusion single photon emission computed tomography (SPECT) intra-observer variability is highly correlated for both regional and global perfusion, with a smaller repeatability coefficient for quantitative analysis.

In a recent study by Chih et al. (9), inter study reproducibility was examined in 20 subjects having two CMRI stress tests an average of 7.7 days apart. The coefficient of variation for repeat testing was 18.9% with a mean difference of 0.07+/-0.26 for MPRI. Similar to our study, this group examined the intra- and inter-observer variation for measurements made using Philips View Forum workstation. They took a subset of their data (a single slice of perfusion data from each subject) for evaluation of the repeatability of the MPRI calculation from their software. They reported the coefficient of variation was 9% for inter observer and 5.3% for intra observer measurements. Correlations were reported using a Spearman correlation coefficient, and were high (0.93 and 0.94 for inter- and intra-observer measurements respectively).

In our study, we focus on the repeatability of measurement made by CAAS MRV 3.3 software (Pie Medical Imaging B.V., Netherlands), and we have included the entire dataset for all 20 scans. Interclass correlation coefficients were high. Variation is lower compared to Chih et al. (9), with coefficient of variation of 7.5% for inter observer and 3.6% for intra observer measurements for the whole group. Our subset analysis in the subjects who were healthy volunteers found coefficient of variation of 9.2%, 3.6% for inter observer and intra observer measurement respectively and lower values were observed for the women with angina and open arteries (5.3% and 3.3% for inter- and intra-observer measurements). Gibbs Ring/Susceptibility artifact was minimized by our use of low dose of gadolinium (0.05 mmol/kg). Furthermore, during post-processing, placement of subendocardial border did not include any dark edge artifact, if present.

There are multiple potential reasons for the differences between our results and those of Chih et al. (9). We have evaluated data using a different software program, although the method of calculation of MPRI as ratio of upslope data adjusted for LV cavity input is the same between these programs. It is possible that the differences in image quality have contributed to greater variation in the measurements made, with our data obtained using a different vendor MRI scanner and a different pulse sequence. Given the variability it is crucial to standardize CMRI data and their analysis in further research studies and in particular for clinical care.

It has been our experience that contour placement is very important and greatly affects the values obtained from software measurement of MPRI. Inclusion of pixels of image data that are not myocardial (blood pool or surrounding epicardial fat) alters the observed values for signal intensity. Limited spatial resolution of the imaging method increases the likelihood that a small adjustment in contour position will cause a significant change in signal intensity values. This “noise” in measurement of the data resulting from the software post processing is a contributing factor to the reported inter-study variation in MPRI.

Improvements in image quality that result from hardware and software advances at the point of image acquisition are likely to result in decreased variance in the measurements made at post processing. Standardization of the user interaction with the software is also important, and having a single user post process all data decreases the variation in measurement further.

Study limitations

This was a small study with 20 studies, so conclusions regarding inter and intra-observer variability may differ with a larger number of studies. This testing was done at an established, high-volume CMRI center where technicians and imaging physicians are highly experienced; the numbers for reproducibility may vary at less experienced centers. Our data is not reporting on variability that is measured in CAD population, as our cohort of women had no obstructive CAD.

Conclusions

It can be concluded from this study that there is measurement variation inherent in the post processing of the perfusion CMR data using standard commercially available software. This variation is minimized by the use of a single observer for post processing data, indicating that the user interaction with the software impacts the values obtained.

Post processing measurement reliability needs to be considered when interpreting results.

Acknowledgements

This work was supported by contracts from the National Heart, Lung and Blood Institutes, nos. N01-HV-68161, N01-HV-68162, N01-HV-68163, N01-HV-68164, a GCRC grant MO1-RR00425 from the National Center for Research Resources, and grants from the Gustavus and Louis Pfeiffer Research Foundation, Denville, New Jersey, the Women’s Guild of Cedars-Sinai Medical Center, Los Angeles, California, the Edythe L. Broad Women’s Heart Research Fellowship, Cedars-Sinai Medical Center, Los Angeles, California, and the Barbra Streisand Women’s Cardiovascular Research and Education Program, Cedars-Sinai Medical Center, Los Angeles.

Disclosure: The authors declare no conflict of interest.

References

1. Klem I, Greulich S, Heitner JF, et al. Value of cardiovascular magnetic resonance stress perfusion testing for the detection of coronary artery disease in women. JACC Cardiovasc Imaging 2008;1:436-45.
2. Klem I, Heitner JF, Shah DJ, et al. Improved detection of coronary artery disease by stress perfusion cardiovascular magnetic resonance with the use of delayed enhancement infarction imaging. J Am Coll Cardiol 2006;47:1630-8.
3. Johnson BD, Shaw LJ, Buchthal SD, et al. Prognosis in women with myocardial ischemia in the absence of obstructive coronary disease: results from the National Institutes of Health-National Heart, Lung, and Blood Institute-Sponsored Women’s Ischemia Syndrome Evaluation (WISE). Circulation 2004;109:2993-9.
4. Pepine CJ. Ischemic heart disease in women. J Am Coll Cardiol 2006;47:S1-3.
5. von Mering GO, Arant CB, Wessel TR, et al. Abnormal coronary vasomotion as a prognostic indicator of cardiovascular events in women: results from the National Heart, Lung, and Blood Institute-Sponsored Women’s Ischemia Syndrome Evaluation (WISE). Circulation 2004;109:722-5.
6. Lanza GA, Buffon A, Sestito A, et al. Relation between stress-induced myocardial perfusion defects on cardiovascular magnetic resonance and coronary microvascular dysfunction in patients with cardiac syndrome X. J Am Coll Cardiol 2008;51:466-72.
7. Panting JR, Gatehouse PD, Yang GZ, et al. Abnormal subendocardial perfusion in cardiac syndrome X detected by cardiovascular magnetic resonance imaging. N Engl J Med 2002;346:1948-53.
8. Pohost GM, Kim RJ, Kramer CM, et al. Task Force 12: training in advanced cardiovascular imaging (cardiovascular magnetic resonance [CMR]) endorsed by the Society for Cardiovascular Magnetic Resonance. J Am Coll Cardiol 2008;51:404-8.
9. Chih S, Macdonald PS, Feneley MP, et al. Reproducibility of adenosine stress cardiovascular magnetic resonance in multi-vessel symptomatic coronary artery disease. J Cardiovasc Magn Reson 2010;12:42.
10. Wei J, Mehta PK, Johnson BD, et al. Safety of coronary reactivity testing in women with no obstructive coronary artery disease: results from the NHLBI-sponsored WISE (Women’s Ischemia Syndrome Evaluation) study. JACC Cardiovasc Interv 2012;5:646-53.
11. Mehta PK, Goykhman P, Thomson LE, et al. Ranolazine improves angina in women with evidence of myocardial ischemia but no obstructive coronary artery disease. JACC Cardiovasc Imaging 2011;4:514-22.
12. Cerqueira MD, Weissman NJ, Dilsizian V, et al. Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart: a statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association. Circulation 2002;105:539-42.
13. Thomson LE, Fieno DS, Abidov A, et al. Added value of rest to stress study for recognition of artifacts in perfusion cardiovascular magnetic resonance. J Cardiovasc Magn Reson 2007;9:733-40.
14. Bland JM, Altman DG. Applying the right statistics: analyses of measurement studies. Ultrasound Obstet Gynecol 2003;22:85-93.
15. Xu Y, Hayes S, Ali I, et al. Automatic and visual reproducibility of perfusion and function measures for myocardial perfusion SPECT. J Nucl Cardiol 2010;17:1050-7.
16. Pennell DJ. Perfusion abnormality, normal coronaries, and chest pain. J Am Coll Cardiol 2008;51:473-5.
17. Shan K, Constantine G, Sivananthan M, et al. Role of cardiac magnetic resonance imaging in the assessment of myocardial viability. Circulation 2004;109:1328-34.
18. Bohl S, Wassmuth R, Abdel-Aty H, et al. Delayed enhancement cardiac magnetic resonance imaging reveals typical patterns of myocardial injury in patients with various forms of non-ischemic heart disease. Int J Cardiovasc Imaging 2008;24:597-607.
19. Hombach V, Grebe O, Merkle N, et al. Sequelae of acute myocardial infarction regarding cardiac structure and function and their prognostic significance as assessed by magnetic resonance imaging. Eur Heart J 2005;26:549-57.
20. Elkington AG, Gatehouse PD, Ablitt NA, et al. Interstudy reproducibility of quantitative perfusion cardiovascular magnetic resonance. J Cardiovasc Magn Reson 2005;7:815-22.
21. Slomka PJ, Alexanderson E, Jácome R, et al. Comparison of clinical tools for measurements of regional stress and rest myocardial blood flow assessed with 13N-ammonia PET/CT. J Nucl Med 2012;53:171-81.