Cardiologist user experience of artificial intelligence-based quantitative coronary angiography

Introduction

Quantitative coronary angiography (QCA) is a guidance tool that quantifies the severity and characteristic features of coronary artery stenosis using two-dimensional static images obtained during coronary angiography (CAG) (1). In addition to the operator’s visual estimation and experience, it provides an objective assessment of coronary artery stenosis and helps with the accurate performance of coronary interventions (2). However, its use in clinical practice has been limited because an experienced staff member needs to operate the workstation and measure the pathologic coronary anatomy.

Therefore, artificial intelligence (AI)-assisted QCA has recently been introduced to address this limitation (3). This new technology enables automated real-time QCA analysis with little need for human intervention. A previous study revealed that AI-QCA demonstrated high sensitivity in lesion detection and strong correlations with manual QCA (4). Furthermore, a previous study has proven its accuracy compared with manual QCA and intravascular ultrasound (5).

This study was, therefore, designed to describe cardiologists’ user patterns and experience with AI-QCA. Interventional cardiologists with varying levels of experience participated in this study and evaluated angiography results, which were prospectively collected. We also compared the time spent in AI-QCA vs. manual QCA, and the angiographic parameters between the two modalities. We present this article in accordance with the STROBE reporting checklist (available at https://cdt.amegroups.com/article/view/10.21037/cdt-2025-269/rc).

Methods

Study design

This prospective study was designed to assess the user experience of interventional cardiologists using AI-QCA software (trial registration number: B-2202-739-303). Ten board-certified cardiologists specializing in coronary intervention with varying periods of experience were enrolled from ten cardiology centers in Korea; informed consent was obtained from all participants. A coronary angiogram was prospectively acquired from 200 patients from the participating centers. The coronary angiograms were randomly assigned to the participating physicians, who were asked to analyze the images using AI-QCA (MPXA-2000, Medipixel Inc., Seoul, Republic of Korea). In addition, a QCA expert performed manual QCA analysis using dedicated software (CAAS, Pie Medical Imaging, Maastricht, Netherlands).

All procedures performed in this study were conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The institutional review board or ethics committee at each participating hospital approved the study protocol for the study. A list of the ethics committee is available upon request.

Study process

CAG images were collected prospectively from patients who underwent CAG and who had provided written consent. Patients were enrolled from nine participating centers. The critical inclusion criterion was stable coronary disease with single-vessel disease (Table S1). Patients with acute coronary syndrome, poor image quality on CAG, stenotic blood vessel lumen diameter less than 2.5 mm, indefinite normal vessel segments, and very long diffuse lesions were excluded. Stratification was applied to ensure the inclusion of minimum case numbers of the left circumflex and right coronary arteries. The organizing committee collected de-identified CAG images, which were sent to one QCA expert. The QCA expert selected the best view and frame and performed manual QCA using CAAS software (Pie Medical Imaging, Maastricht, Netherlands). The analysis time was recorded using a stopwatch.

Coronary angiograms were then randomly allocated to the study physicians. While angiograms from a center were not assigned to a physician from the same center, simple random sampling was done with no stratification. Cardiologists were requested to evaluate the angiogram images using the MPXA-2000 software (Figure 1). Information on the best angle and frame selected by the manual QCA analyst was provided; however, the AI-QCA-assisted analysis was fully at the discretion of the cardiologists. Prior to analysis, Digital Imaging and Communications in Medicine (DICOM) files must be uploaded via the web-based AI-QCA platform. The optimal frame for QCA analysis is automatically recommended by the system. AI-QCA is performed to identify the most severe lesion, with P and D representing proximal and distal reference segments, respectively. Lumen diameter measurements along the vessel length are calculated and displayed. Coronary branch analysis results are presented (Figure S1). The editing of lesion and vessel contours was allowed. Furthermore, usage patterns, such as the selection of view angle and frame, and analysis time extracted, were collected from log data.

Figure 1 Screen captures of a representative case assessment using artificial intelligence-assisted quantitative coronary analysis software. CAU, caudal; CF, calibration factor; CRA, cranial; DS, diameter stenosis; LAD, left anterior descending artery; LAO, left anterior oblique; LCX, left circumflex artery; LM, left main coronary artery; MLD, minimal lumen diameter; RAO, right anterior oblique; RCA, right coronary artery; RD, reference vessel diameter.

System Usability Scale (SUS)

The usage patterns and experience with using the MPXA-2000 algorithm were evaluated through a questionnaire based on the SUS, comprising ten questions, which were presented to each expert (6). SUS is considered as a reliable and valid measure of perceived usability. Questionnaires were completed online by the ten study cardiologists, and the acceptability of the AI-QCA results was graded as not acceptable, marginal low, marginal high, or acceptable, as previously described as an empirical evaluation (7).

Angiographic parameters

QCA variables included minimum luminal diameter, reference diameter, proximal reference diameter to the stenotic lesion, distal reference diameter to the stenotic lesion, percent diameter stenosis, and lesion length (8). The sample size for this study was determined empirically, given its exploratory nature, focusing on physicians’ user experience, rather than testing a pre-specified statistical hypothesis.

Statistical analysis

The relationship between AI-QCA and manual QCA was investigated by creating scatter plots and performing Bland-Altman analysis to assess agreement between AI-QCA and manual QCA, with correlation coefficients calculated. However, the correlation between AI-QCA and manual QCA was assessed using Pearson’s or Spearman’s correlation coefficients, depending on the normality of the distribution of the tested variables, as determined using the Shapiro-Wilk test. Correlation strength between AI-QCA and manual QCA measurements was interpreted as very strong (R=0.90–1.00), strong (R=0.70–0.89), moderate (R=0.40–0.69), weak (R=0.10–0.39), and negligible (R=0.00–0.10) (9). Statistical significance was set at two-sided P values <0.05, and all statistical analyses were conducted using R programming version 4.4.1 (The R Foundation for Statistical Computing).

Results

Characteristics of participating physicians and patients

Ten board-certified interventional cardiologists, with varying degrees of experience, participated: five had ≥10 years of experience as an independent operator, four had 5–9 years of experience, and one had <5 years of experience. Overall, 193 patients were enrolled between February 2022 and December 2023. Angiograms of eleven patients were excluded from the analysis as three had total occlusions, two had vessel diameters <2.5 mm, two had calibration factor errors, three had extreme anatomical conditions such as aneurysms, and one was not analyzable due to poor image quality. Two patients were excluded from statistical analysis due to measurements that exceeded the predetermined threshold established in the study protocol. Therefore, the angiograms of 180 patients were finally analyzed (Figure S2). Table 1 shows the baseline characteristics of the study population. Their mean age was 64.9±9.9 years, and 76.5% were male. The study design mandated that every patient had a single-vessel disease. The distribution of stenotic lesions was 65.8% in the left anterior descending artery; 12.4% in the left circumflex artery; and 21.8% in the right coronary artery. The average percent diameter stenosis, reference diameter, and lesion length were 70%, 2.9 mm, and 22 mm, respectively.

User experience and system usability scale of AI-QCA

Cardiologists that used AI-QCA selected the same view as the manual QCA analyst for 146 of the 180 (81.1%) angiograms. Angiographic frames selected by cardiologists using AI-QCA were within five frames of those chosen by manual QCA analysts in 65% (117/180) of cases, demonstrating similar frame selection patterns between the two methods. Furthermore, the time spent on AI-QCA for total cohort (n=180) was significantly lower than that for manual QCA (1.5±0.9 vs. 88.1±35.5 s, P<0.001) (Figure 2).

Figure 2 Time spent on AI-assisted QCA and manual QCA. AI, artificial intelligence; QCA, quantitative coronary angiography.

The average SUS score among the participating cardiologists was 66.7, indicating marginally high acceptability (one cardiologist was excluded due to relocation to another institution). Notably, 3 (33.3%), 3 (33.3%), 1 (11.1%), and 2 (22.2%) of the nine respondents considered the system acceptable, marginally high, low marginal, and not acceptable, respectively. The mean SUS score was 73.2 after excluding the two outliers, which indicated an acceptable user experience.

Angiographic measurements using AI-QCA and manual QCA

Figures 3,4 show the scatterplots and Bland-Altman analysis for angiographic variables obtained using AI-QCA and manual QCA for the total cohort (n=180). Among the measured parameters, distal reference diameter to stenotic lesion (DRD) showed a strong correlation between the two QCA methods, whereas minimal lumen diameter (MLD), reference diameter (RD), proximal reference diameter (PRD), diameter stenosis (DS), and lesional length (LL) demonstrated moderate correlations. Furthermore, sensitivity analyses with 146 angiograms in which the same view was selected between the two modalities showed similar results (Figures S3,S4).

Figure 3 Scatter plot showing correlation of six measured QCA parameters between manual QCA and AI-QCA: (A) MLD, (B) RD, (C) PRD, (D) DRD, (E) percent DS, and (F) LL. AI, artificial intelligence; DRD, distal reference diameter to stenotic lesion; DS, diameter stenosis; LL, lesional length; MLD, minimum lumen diameter; PRD, proximal reference diameter to stenotic lesion; QCA, quantitative coronary angiography; RD, reference vessel diameter.

Figure 4 Bland-Altman plots demonstrating agreement between manual and AI-QCA measurements for six parameters in the subset with matching angiographic view selections: (A) MLD, (B) RD, (C) PRD, (D) DRD, (E) percent DS, and (F) LL. AI, artificial intelligence; DRD, distal reference diameter to stenotic lesion; DS, diameter stenosis; LL, lesional length; MLD, minimum lumen diameter; PRD, proximal reference diameter to stenotic lesion; QCA, quantitative coronary angiography; RD, reference vessel diameter.

Discussion

This prospective study evaluated cardiologists’ experience using AI-QCA. Participating cardiologists generally reported satisfactory user experience and acceptable usability. Cardiologists accepted the frame number and made few changes to the lesion segmentation in approximately two-thirds of the angiograms performed using the AI-QCA algorithm. Therefore, our results revealed that the time for lesion analysis was significantly reduced with the use of AI-QCA compared with manual QCA. Lesion analysis with AI-QCA required minimal human intervention, and critical parameters on coronary plaque analyses were similar to those obtained with manual analyses.

Analysis on CAG poses inherent challenges such as cardiac motion blur, the need for radiation and radiocontrast dye, and radiographic attenuation by anatomical structures (e.g., ribs and vertebrae). AI has been increasingly utilized in clinical medicine as it helps humans perform specific tasks, and reduces repetitive work (10). Notably, recent studies have attempted automated coronary analysis using advanced machine learning techniques (10-13). While accuracy and interobserver variability have been tested, no previous studies have evaluated real-world acceptance by clinicians.

However, to our knowledge, the AI-QCA system tested in this study is the first commercialized product for CAG analysis. The core algorithm is based on deep learning networks trained on angiographic images of 3,302 diseased major vessels from 2,042 patients (3). The final product further incorporates an ensemble architecture that integrates three neural networks for semantic segmentation (U-Net++, U2-Net, and DeepLabV3+) (3,5). Kim et al. showed that AI-QCA presents a sensitivity of 89% in lesion detection and strong correlations with manual QCA (4). Furthermore, Moon et al. compared AI-QCA measurements of coronary artery lesions with conventional invasive measurement tools such as intravascular ultrasound, reporting a moderate-to-strong correlation with IVUS in analyzing coronary lesions with significant stenosis (5). In addition, a randomized controlled trial demonstrated that AI-QCA-assisted coronary intervention was not inferior to intervention guided by optical coherence tomography in terms of minimal stent area (14,15).

Our results demonstrated an acceptable user experience among interventional cardiologists using the AI-QCA software. Notably, the biggest advantage of using the software was the reduced analysis time. The additional processing time required for AI-QCA was negligible when compared to manual QCA, requiring approximately 1.5 min per analysis. Another advantage was flexibility. Users could freely choose an appropriate angle and frame for quantitative analysis, which did not differ remarkably from that performed by a dedicated analyst. The perceived usability varied widely across respondents, whereas the average SUS score indicated marginally high acceptability. This may be attributable to the high flexibility of the platform. Notably, a user can select many options and tools to optimize angiographic analysis. Furthermore, some cardiologists may accept it easily, whereas others, who are not familiar with the traditional QCA process, may find it complicated or difficult to learn. There were two outliers out of nine participants, who apparently had trouble handling the web-based software and gave extremely low scores. In the meantime, there was a small variation in scores across the 10 questions. After excluding the two outliers, the mean SUS score indicated an acceptable user experience.

This study offers insights into integrating AI-QCA into clinical practice. The AI-assisted technology has the potential to reduce healthcare personnel’s time by offering consistent lesion identification and analysis. However, our findings also highlight areas for improving user experience. Workflow convenience is crucial for physicians’ adoption, particularly in streamlining the process from CAG image acquisition to automated lesion analysis. While steps such as sign-in, image selection, frame selection, segmentation contour confirmation were necessary for this study, they may have impeded physicians’ user experience. Additional potential barriers include the need for training and associated costs. It needs to be tested in future studies if improvements in user interface, training, or integration into clinical workflow would enhance user experience.

This study has some limitations. The sample size was relatively small, and the study cardiologists were recruited from a single geographic region. Patients with simple, stable coronary lesions were enrolled in this study; cases involving total occlusion or diffuse coronary lesions were excluded. Therefore, the study’s findings are not generalizable to more complex lesions or acute coronary syndrome. SUS is widely used and technologically-agnostic; however, it omits essential information specific to an interface type (16). As number of participating cardiologists were small, the average SUS score result may not represent universal user experiences. Furthermore, although we assessed user experience on the web version of the software, further studies should test how clinical implementation may help physicians perform coronary angiographies and interventions.

Conclusions

This prospective study showed that automated QCA could be performed with minimal human intervention and reduced time, compared with traditional manual QCA. Participating cardiologists perceived the usability of the software as marginally highly acceptable. Angiographic parameters acquired with the assistance of AI-QCA were comparable to those acquired with manual QCA. Therefore, the application of AI-QCA in the Cath lab may be feasible and helpful during coronary angiographies and interventions.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://cdt.amegroups.com/article/view/10.21037/cdt-2025-269/rc

Data Sharing Statement: Available at https://cdt.amegroups.com/article/view/10.21037/cdt-2025-269/dss

Peer Review File: Available at https://cdt.amegroups.com/article/view/10.21037/cdt-2025-269/prf

Funding: This study was financially supported by Medipixel Inc. and the Bio&Medical Technology Development Program of the National Research Foundation (NRF) funded by the Korean government (MSIT) (No. RS-2023-00222910).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://cdt.amegroups.com/article/view/10.21037/cdt-2025-269/coif). All authors report that Medipixel, Inc. has supported this study. SHK owns stock options of Medipixel, Inc. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The institutional review board or ethics committee at each participating hospital approved the study protocol for the study. Informed consent was obtained from all participants.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Dangas GD, Di Mario C, Thiele H, et al. editors. Interventional cardiology: principles and practice. West Sussex, UK: Wiley-Blackwell; 2022.
2. Garrone P, Biondi-Zoccai G, Salvetti I, et al. Quantitative coronary angiography in the current era: principles and applications. J Interv Cardiol 2009;22:527-36. [Crossref] [PubMed]
3. Yang S, Kweon J, Roh JH, et al. Deep learning segmentation of major vessels in X-ray coronary angiography. Sci Rep 2019;9:16897. [Crossref] [PubMed]
4. In Kim Y, Roh JH, Kweon J, et al. Artificial intelligence-based quantitative coronary angiography of major vessels using deep-learning. Int J Cardiol 2024;405:131945. [Crossref] [PubMed]
5. Moon IT, Kim SH, Chin JY, et al. Accuracy of Artificial Intelligence-Based Automated Quantitative Coronary Angiography Compared to Intravascular Ultrasound: Retrospective Cohort Study. JMIR Cardio 2023;7:e45299. [Crossref] [PubMed]
6. Lewis JR. The system usability scale: past, present, and future. International Journal of Human-Computer Interaction. 2018;34:577-90. [Crossref]
7. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Intl Journal of Human-Computer Interaction 2008;24:574-94. [Crossref]
8. Tomasello SD, Costanzo L, Galassi AR. Quantitative coronary angiography in the interventional cardiology. In: Kirac S. editor. Advances in the Diagnosis of Coronary Atherosclerosis. Rijeka, Croatia: InTech; 2011.
9. Schober P, Boer C, Schwarte LA. Correlation Coefficients: Appropriate Use and Interpretation. Anesth Analg 2018;126:1763-8. [Crossref] [PubMed]
10. Danilov VV, Klyshnikov KY, Gerget OM, et al. Real-time coronary artery stenosis detection based on modern neural networks. Sci Rep 2021;11:7582. [Crossref] [PubMed]
11. Yabushita H, Goto S, Nakamura S, et al. Development of Novel Artificial Intelligence to Detect the Presence of Clinically Meaningful Coronary Atherosclerotic Stenosis in Major Branch from Coronary Angiography Video. J Atheroscler Thromb 2021;28:835-43. [Crossref] [PubMed]
12. Labrecque Langlais É, Corbin D, Tastet O, et al. Evaluation of stenoses using AI video models applied to coronary angiography. NPJ Digit Med 2024;7:138. [Crossref] [PubMed]
13. Avram R, Olgin JE, Ahmed Z, et al. CathAI: fully automated coronary angiography interpretation and stenosis estimation. NPJ Digit Med 2023;6:142. [Crossref] [PubMed]
14. Kim Y, Park H, Yoon HJ, et al. Fully automated quantitative coronary angiography versus optical coherence tomography guidance for coronary stent implantation (FLASH): Study protocol for a randomized controlled noninferiority trial. Am Heart J 2024;275:86-95. [Crossref] [PubMed]
15. Kim Y, Yoon HJ, Suh J, et al. Artificial Intelligence-Based Fully Automated Quantitative Coronary Angiography vs Optical Coherence Tomography-Guided PCI: The FLASH Trial. JACC Cardiovasc Interv 2025;18:187-97. [Crossref] [PubMed]
16. Sauro J. SUPR-Q: A comprehensive measure of the quality of the website user experience. Journal of Usability Studies 2015;10:68-86.