Accuracy of wearable heart rate monitors in cardiac rehabilitation

Introduction

A multidisciplinary cardiac rehabilitation (CR) program that includes supervised exercise training improves long-term survival after cardiac events (e.g., myocardial infarction, coronary revascularization), and structured exercise programs improve outcomes in patients with heart failure (1-5). Recognizing the well-documented clinical benefits of CR, ACC/AHA and European guidelines strongly recommend CR after myocardial infarction and coronary revascularization and structured exercise in patients with heart failure (6,7). Despite the clinical benefits of CR, only 20% to 30% of eligible patients complete a CR program (3,6-8).

The requirement that patients travel to a CR center has been cited as a barrier to CR (9). To address this challenge, centers have developed in-home CR programs and “telerehabilitation” programs, the latter employing electronic communication and/or remote monitoring as a component of in-home CR; such approaches have been shown to have efficacy equal to that of center-based CR (10-14). However, in-home CR requires the development of novel strategies to monitor CR-recommended exercise and ensure patient safety.

Heart rate (HR) monitoring is a crucial component of structured CR programs, and target HRs are established to guide patient-specific exercise intensities (15,16). In hospital-based programs, this is often accomplished using ECG telemetry monitoring, a modality that has long been considered the “gold standard” for monitoring during CR (17-19). More recently, the use of electrode-based chest strap monitors has been employed in CR programs and in clinical research trials (20). Neither of these techniques for HR monitoring is convenient for patients at home. For example, elderly patients may be uncomfortable with the relative cumbersome application of chest strap monitors and establishing the Bluetooth connection necessary to display and record HR. Therefore, some investigators have proposed that in-home CR exercise sessions be monitored with commercially-available, optically based wrist-worn HR monitors (15,16).

Recent studies in healthy volunteers suggest that the accuracy of popular wrist-worn, optically based HR monitors varies between devices and is influenced by activity type and intensity (21,22). The accuracy of wrist-worn monitors has not been systematically assessed in patients with cardiovascular disease, particularly in those who use them in the context of CR. If data from these monitors are to be used to guide patient activity and therapy, the monitors’ accuracy must be validated.

The objective of this study was to assess the accuracy of four commonly used, commercially available, optically based wearable HR monitors in patients with established cardiovascular disease enrolled in a CR program at a tertiary care center.

Methods

Participants

This prospective study recruited 80 adults aged 18 years or older enrolled in a Phase II or III CR program at a tertiary care academic medical center. Phase II CR is early outpatient treatment that occurs after cardiac events and procedures; it is performed under close supervision and includes HR monitoring. Phase II CR may extend for up to 36 sessions over the course of approximately 12 weeks (17,18). Phase III CR also occurs in an outpatient setting and seeks to promote long-term engagement in exercise and a healthy lifestyle; phase III CR follows Phase II CR and has less supervision than does Phase III CR (17-19). Like most centers, we employ HR monitoring to guide exercise intensity for both phase II and phase III CR (17-19).

Study exclusion criteria included a cardiac pacemaker, tattoos around the wrist or forearm, and use of a radial artery for coronary artery bypass grafting. The protocol was approved by the Institutional Review Board of the Cleveland Clinic and all subjects provided written informed consent. The study was registered prospectively at clinicaltrials.gov (NCT03182439).

HR monitors

All participants wore standard electrocardiographic (ECG) leads (Mason-Likar electrode placement of torso-mounted limb leads) and a Polar H7 chest strap monitor (Polar, Inc., Kempele, Finland). In addition, each participant was randomly assigned by a computer program to wear two different wrist-worn HR monitors, one on each wrist; this enabled assessment of each type of wrist-worn monitor in 40 subjects. The wrist-worn monitors assessed included the Fitbit Blaze (Fitbit Inc., San Francisco, CA, USA), Apple Watch (Apple Inc., Cupertino, CA, USA), Garmin Forerunner 235 (Garmin Inc., Olathe, KS, USA) and TomTom Spark Cardio (TomTom, Inc., Amsterdam, Netherlands). Four units of each type of monitor were purchased from retail outlets and studied in random order.

Exercise protocol

In each subject, HR was assessed using 4 different monitoring modalities (ECG, Polar H7 chest strap, and two different wrist-worn monitors). Wrist-worn monitors were affixed securely above the ulnar styloid. The Mason-Likar electrode placement allowed the assessment of modified leads I, II, and III on ECG. An aggressive electrode preparation was performed at each site, which included cleansing with alcohol and light abrasion to reduce resistance and optimize signal quality. ECG was monitored on a Quinton Q-tel RMS telemetry system (Quinton Cardiology Systems Inc., Bothell, WA, USA), and hard copy rhythm strips were obtained to measure ECG. ECG-based HR was determined by visual assessment under direct supervision by a cardiologist; ECG-based HR was able to be ascertained at all-time points, and ECG artifact was not observed. Patients were assessed in two different exercise conditions: treadmill and stationary cycle. For both the treadmill and the stationary cycle, HR was measured at rest and at 3, 5, and 7 minutes of individually prescribed steady-state exercise at 50–70% of HR reserve. This provided HR measurements at 8 time points for each participant.

HR signals for all devices were checked at the beginning of each exercise/rest segment to ensure device function. At each time point, HRs were recorded by 2 trained research personnel (M Etiwy, Z Akhrass, L Gillinov), one situated on each side of the subject. HR recordings from all devices and ECG were obtained over a period of approximately 5 seconds. Values were entered into an IRB-approved database.

Statistical methods

Sample size

Sample size was based on the use of Lin’s concordance correlation coefficient (r_c) to compare HR measurements obtained with wearable, optically based HR monitors to those obtained with the ECG, which is considered the standard (23). To detect a difference in r_c of .85 under H0 and 0.94 under H1, a sample of 40 pairs of subjects results in 90% power using a one-sided z test with 0.05 significance level (21,22). Therefore, with a randomized block design and each patient testing 2 of the 4 monitors, a sample size of 80 patients was deemed necessary.

Analysis plan

Paired differences

Using the ECG-determined HR as the standard, each of the HR monitoring systems was assessed for accuracy by calculating the difference between the measures and comparing. The paired differences were calculated as (HR_ecg – HR_device) for each device under the various conditions. Absolute values of the paired differences were also summarized to better assess the magnitude of difference irrespective of direction. Percent differences were calculated as [(HR_ecg – HR_device)/HR_ecg]×100.

Agreement

Bland-Altman analysis was performed to assess agreement for each device with ECG (24). In addition, Lin’s concordance correlation coefficients (r_c) and associated 95% confidence intervals were calculated to provide a measure of agreement for each device with ECG. The concordance correlation coefficient (r_c) measures the degree to which the paired observations fall on the identity line. An (r_c) >0.8 was deemed to represent acceptable accuracy in HR measurement (18).

Multivariable testing

Repeated-measures mixed-model analysis of variance was used to test the overall effect of the monitoring devices while adjusting for covariates and taking into account multiple measurements for each subject. In addition to HR device and exercise condition, factors considered in the multivariable model included age, gender, body mass index, wrist size, medications, skin color and presence of coronary artery disease, hypertension, diabetes, and atrial fibrillation. A criterion of P<0.05 was used for retention of factors in the final adjusted model, with HR device and exercise condition retained regardless of significance.

Data were analyzed using SAS version 9.4 (SAS Institute, Inc., Cary, NC, USA) and R software version 3.2.3.

Presentation

Continuous variables are reported as mean ± standard deviation and categorical variables as frequency and percentage.

Results

Patients

The study randomized 80 patients, most of whom were engaged in Phase II CR as a component of management of their coronary artery disease (Table 1). There were no significant differences in the characteristics of patients randomized to different HR monitors.

Table 1 Patient characteristics (n=80)
Full table

Aggregate results

Of the 2,560 possible HR measurements [80 subjects, 8 time points, 4 devices per subject (ECG, Polar chest strap, two wrist-worn monitors)], 2,546 were recorded (99.5%). Missing data were attributable to failure of the device to display/record HR (5 for Apple Watch and 9 for TomTom Spark Cardio).

Measured HR ranged from 51 to 165 beats per minute (bpm). Average absolute differences from the ECG standard were less than 1 bpm for the Polar H7 under all exercise conditions and ranged from less than 1 bpm to nearly 13 bpm for other monitors (Table 2).

Table 2 Heart rate (bpm) monitor differences from ECG according to activity
Full table

Bland-Altman analysis revealed that all monitors had some measurements that did not reflect HR accurately (Figure 1); however, this variation was not linked to specific HR values, meaning that variability was not influenced by the HR magnitude. The Apple Watch had 95% of differences fall within −24 and 23 bpm of the ECG (mean difference −0.59 bpm), while TomTom Spark Cardio and Garmin Forerunner 235 had 95% of values fall within −25 and 25 bpm (mean difference −0.02 bpm) and −33 and 40 bpm (mean difference 3.8 bpm), respectively. The corresponding values for Fitbit Blaze were –22 and 28 bpm (mean difference 2.9 bpm).

Figure 1 Bland-Altman plots and 95% limits of agreement with electrocardiographically measured heart rate. (A) Polar H7; (B) Apple Watch; (C) Fitbit Blaze; (D) Garmin Forerunner 235; (E) TomTom Spark Cardio.

Under all conditions combined, when compared to ECG, the Polar chest strap had the highest agreement with ECG with an r_c of 0.99. Among wrist-worn monitors, the Apple Watch performed best with r_c=0.80, followed by the Fitbit Blaze (r_c=0.78), TomTom Spark Cardio (r_c=0.76) and Garmin Forerunner 235 (r_c=0.52) (Figure 2).

Figure 2 Concordance correlation coefficients depicting agreement of device-measured heart rate with electrocardiogram. (A) Polar H7; (B) Apple Watch; (C) Fitbit Blaze; (D) Garmin Forerunner 235; (E) TomTom Spark Cardio.

Results of the mixed model confirmed that among the wrist-worn HR monitors, the Apple Watch and TomTom Spark Cardio were most accurate, with no statistical difference from ECG (P=0.62 for TomTom Spark Cardio and P=0.09 for Apple Watch), even after adjustment for all other factors. The other optically based HR monitors often underestimated the true HR (P<0.0002). Overall, wrist-worn HR monitors were less accurate in younger patients (P=0.0002) and in those taking diuretics (P=0.0009); other factors, including wrist circumference, body mass index, gender, skin color, presence of atrial fibrillation, and additional medication use did not influence HR monitor accuracy.

Agreement with ECG during various types of exercise

The Polar H7 chest strap performed well in all conditions (rest, treadmill, stationary cycle, r_c>0.99 for all conditions), but other HR monitors’ agreement with ECG varied with the type of exercise (Table 2). At rest, all monitors had r_c>0.80 with the exception of the Garmin Forerunner 235 (r_c=0.60). With the treadmill, the Fitbit Blaze (r_c=0.76) and the Apple Watch performed best (r_c=0.70), while the Garmin Forerunner 235 and TomTom Spark Cardio were less accurate (r_c=0.53 for each). With the stationary cycle, the Apple Watch (r_c=0.89), TomTom Spark Cardio (r_c=0.85) and Fitbit Blaze performed best (r_c=0.72), while the Garmin Forerunner 235 was less accurate (r_c=0.39).

Discussion

This study demonstrates that optically based, wrist-worn HR monitors are less accurate than electrode-containing chest strap monitors in patients with cardiovascular disease engaged in standard CR exercise protocols. Accuracy varied by monitor type and, to a lesser extent, under different exercise conditions. Wrist-worn monitors tended to underestimate HR, and all periodically delivered spurious HR values. These findings raise concerns regarding the role of wrist-worn HR monitors in CR programs.

Commercially available HR monitors

Introduced in the 1980s, chest-strap-based HR monitors function much like an ECG, sensing cardiac electrical activity. Several studies confirm the accuracy of most of these HR monitors under conditions of both rest and moderate exercise (25). Because chest-strap-based HR monitors are somewhat inconvenient, they have not been widely adopted by the public. In contrast, the recent introduction of convenient, wrist-worn HR monitors that include the capability for wireless transmission has stirred widespread public interest in HR monitoring. The new wrist-worn HR monitors do not measure cardiac electrical activity; rather, they rely on photoplethysmography. The monitor illuminates the skin with a light-emitting diode and then measures the amount of light reflected back to a photodiode sensor; this enables detection of variations in blood volume associated with the pulse of blood caused by each cardiac contraction. Potential sources of error with optically based monitors include motion artifact from physical movement, misalignment between the skin and the optical sensor, variations in skin color/tone, ambient light, peripheral edema and poor tissue perfusion, which may be associated with cardiovascular disease (26).

The accuracy of such monitors during exercise is controversial, with some studies suggesting that wrist-worn HR monitors perform best at rest or slow walking and others documenting good accuracy even during vigorous exercise (20,21,27). Type of activity (e.g., walking vs. elliptical training vs. cycling) influences monitors’ accuracy (22). Wrist-worn HR monitors exhibit a general tendency to underestimate HR, a feature that could create risk for cardiac patients with a specified HR target (28). In addition, recent studies of healthy young individuals reveal considerable variability in accuracy between different monitors; although all function according to similar principles, proprietary differences in technology and algorithms for signal processing likely explain these differences (21,22,28).

Patients with cardiovascular disease

To date, studies have not assessed the accuracy of wearable HR monitors in patients with cardiovascular disease. These patients present a variety of potential challenges to monitors’ accuracy, including hypertension, peripheral arterial disease, venous insufficiency, obesity, atrial fibrillation, and use of medications that affect HR, vascular tone, and volume status (e.g., beta-blockers, angiotensin-converting enzyme inhibitors, calcium channel blockers, and diuretics). Therefore, one cannot assume that monitors proven accurate in healthy individuals will display similar accuracy in those with cardiovascular disease.

In this study, accuracy of wrist-worn HR monitors was substantially lower than the accuracy of the electrically based chest strap monitors. In addition, most of the monitors were less accurate in CR patients than has been previously reported in healthy volunteers (21,22,28). Although wrist-worn monitors generally provided values within ten percent of the actual HR in CR patients, Bland-Altman analysis revealed that more than five percent of measurements were off by at least 20 bpm for all monitors tested. This suggests that the presence of cardiovascular disease may have an important influence on monitors’ accuracy and confirms the need to validate these monitors in such patients before using them to guide therapy.

The precise accuracy of HR monitors that is necessary to guide exercise intensity in CR and ensure patient safety is currently unknown. From a clinical perspective, it certainly seems logical that patients will derive the greatest benefit from CR if they are able to exercise to their prescribed, target HR. In addition the finding that a substantial percentage of HR measurements (5%) were off by a large margin is cause for concern. Patients’ recognition of wildly inaccurate HR measurements could provoke anxiety and might also limit compliance with CR.

Among patients with cardiovascular disease, atrial fibrillation and beta-blocker use did not appear to influence monitors’ accuracy; however, because only small proportions of patients in this study were not treated with beta-blockers or had atrial fibrillation, this observation requires validation. Diuretic use did appear to reduce accuracy, perhaps by influencing circulating blood volume; this finding, along with the observation of reduced accuracy in younger patients, requires further study.

Limitations

Although this study is the largest of its kind in patients with cardiovascular disease and included more than 2,500 HR measurements, it has limitations. The results apply only to the HR monitors tested. These monitors were chosen because of their documented popularity with the public, and each monitor was the manufacturer’s most recent offering at the time of the study; however, they represent an incomplete sample of the wide range of available HR monitors. The study methodology (visual recording of HR on ECG) may have contributed to some error as compared with a more rigorous approach wherein time-stamped raw device data are extracted or HR is presented as a continuous variable; however, at this time continuous HR assessment and raw data capture are not possible with all devices. The devices were assessed in 80 patients undergoing CR; however, their mean age was relatively low (62±13 years), and only a minority were female (19%). Further investigation is warranted in both women and older individuals. Although we accounted for medication use among participants, the precise impact (if any) of specific medications requires examination of larger numbers of patients. Finally, this study assessed the accuracy of wrist-worn monitors in a hospital-based CR facility, as such a setting facilitated a controlled trial in cardiac patients; the safety of such HR monitors in the setting of in-home CR was not assessed.

Conclusions

Novel programs designed to enable CR in the home promise to increase participation in this clinically important therapy. Such efforts require the application of validated methods to measure HR. The U.S. Food and Drug Administration currently considers wrist-worn HR monitors and other wearables as class I (low-risk) devices, obviating legal requirements for approval and regulation. However, clinicians recognize the possibility that data derived from such devices are likely to play an increasingly important role in clinical decision-making. Wrist-worn HR monitors have variable accuracy in CR patients, and this accuracy is lower than that previously reported in patients without cardiovascular disease. Assessment of the clinical significance of these findings is necessary before clinicians can be assured of the safety of relying on wrist-worn HR monitors to guide clinical decision-making and in-home CR.

Acknowledgments

Supported by The Mary Elizabeth Holdsworth Fund at the Cleveland Clinic.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The protocol was approved by the Institutional Review Board of the Cleveland Clinic and all subjects provided written informed consent.

References

1. Lawler PR, Filion KB, Eisenberg MJ. Efficacy of exercise-based cardiac rehabilitation post-myocardial infarction: A systematic review and meta-analysis of randomized controlled trials. Am Heart J 2011;162:571-84.e2. [Crossref] [PubMed]
2. Dunlay SM, Pack QR, Thomas RJ, et al. Participation in cardiac rehabilitation, readmissions, and death after acute myocardial infarction. Am J Med 2014;127:538-46. [Crossref] [PubMed]
3. Ades PA, Keteyian SJ, Wright JS, et al. Increasing cardiac rehabilitation participation from 20% to 70%: A road map from the million hearts cardiac rehabilitation collaborative. Mayo Clin Proc 2017;92:234-42. [Crossref] [PubMed]
4. Smith SC Jr, Benjamin EJ, Bonow RO, et al. AHA/ACCF secondary prevention and risk reduction therapy for patients with coronary and other atherosclerotic vascular disease: 2011 update: A guideline from the American heart association and American college of cardiology foundation endorsed by the world heart federation and the preventive cardiovascular nurses association. J Am Coll Cardiol 2011;58:2432-46. [Crossref] [PubMed]
5. Yancy CW, Jessup M, Bozkurt B, et al. 2013 ACCF/AHA guideline for the management of heart failure: A report of the American college of cardiology foundation/American heart association task force on practice guidelines. J Am Coll Cardiol 2013;62:e147-239. [Crossref] [PubMed]
6. Piepoli MF, Hoes AW, Agewall S, et al. 2016 European guidelines on cardiovascular disease prevention in clinical practice: The sixth joint task force of the European society of cardiology and other societies on cardiovascular disease prevention in clinical practice (constituted by representatives of 10 societies and by invited experts) developed with the special contribution of the European association for cardiovascular prevention & rehabilitation (EACPR). Eur Heart J 2016;37:2315-81. [Crossref] [PubMed]
7. Amsterdam EA, Wenger NK, Brindis RG, et al. 2014 AHA/ACC guideline for the management of patients with non-ST-elevation acute coronary syndromes: Executive summary: A report of the American college of cardiology/American heart association task force on practice guidelines. Circulation 2014;130:2354-94. [Crossref] [PubMed]
8. Kachur S, Chongthammakun V, Lavie CJ, et al. Impact of cardiac rehabilitation and exercise training programs in coronary heart disease. Prog Cardiovasc Dis 2017;60:103-14. [Crossref] [PubMed]
9. Ruano-Ravina A, Pena-Gil C, Abu-Assi E, et al. Participation and adherence to cardiac rehabilitation programs. A systematic review. Int J Cardiol 2016;223:436-43. [Crossref] [PubMed]
10. Brouwers RW, Kraal JJ, Traa SC, et al. Effects of cardiac telerehabilitation in patients with coronary artery disease using a personalised patient-centred web application: Protocol for the smartcare-cad randomised controlled trial. BMC Cardiovasc Disord 2017;17:46. [Crossref] [PubMed]
11. Kraal JJ, Vromen T, Spee R, et al. The influence of training characteristics on the effect of exercise training in patients with coronary artery disease: Systematic review and meta-regression analysis. Int J Cardiol 2017;245:52-8. [Crossref] [PubMed]
12. Piotrowicz E, Korzeniowska-Kubacka I, Chrapowicka A, et al. Feasibility of home-based cardiac telerehabilitation: Results of teleintermed study. Cardiol J 2014;21:539-46. [Crossref] [PubMed]
13. Varnfield M, Karunanithi M, Lee CK, et al. Smartphone-based home care model improved use of cardiac rehabilitation in postmyocardial infarction patients: Results from a randomised controlled trial. Heart 2014;100:1770-9. [Crossref] [PubMed]
14. Clark RA, Conway A, Poulsen V, et al. Alternative models of cardiac rehabilitation: A systematic review. Eur J Prev Cardiol 2015;22:35-74. [Crossref] [PubMed]
15. Kakria P, Tripathi NK, Kitipawang P. A real-time health monitoring system for remote cardiac patients using smartphone and wearable sensors. Int J Telemed Appl 2015;2015:373474. [Crossref] [PubMed]
16. Medina Quero J, Fernández Olmo MR, Peláez Aguilera MD, et al. Real-time monitoring in home-based cardiac rehabilitation using wrist-worn heart rate devices. Sensors (Basel) 2017. [Crossref] [PubMed]
17. Verrill D, Ashley R, Witt K, et al. Recommended guidelines for monitoring and supervision of North Carolina phase ii/iii cardiac rehabilitation programs. A position paper by the North Carolina cardiopulmonary rehabilitation association. J Cardiopulm Rehabil 1996;16:9-24. [Crossref] [PubMed]
18. Sebastian LA, Reeder S, Williams M. Determining target heart rate for exercising in a cardiac rehabilitation program: A retrospective study. J Cardiovasc Nurs 2015;30:164-71. [Crossref] [PubMed]
19. Brawner CA, Girdano D, Ehrman JK, et al. Association between phase 3 cardiac rehabilitation and clinical events. J Cardiopulm Rehabil Prev 2017;37:111-8. [Crossref] [PubMed]
20. O'Connor CM, Whellan DJ, Lee KL, et al. Efficacy and safety of exercise training in patients with chronic heart failure: Hf-action randomized controlled trial. JAMA 2009;301:1439-50. [Crossref] [PubMed]
21. Wang R, Blackburn G, Desai M, et al. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol 2017;2:104-6. [Crossref] [PubMed]
22. Gillinov S, Etiwy M, Wang R, et al. Variable accuracy of wearable heart rate monitors during aerobic exercise. Med Sci Sports Exerc 2017;49:1697-703. [Crossref] [PubMed]
23. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989;45:255-68. [Crossref] [PubMed]
24. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307-10. [Crossref] [PubMed]
25. Léger L, Thivierge M. Heart rate monitors: Validity, stability, and functionality. Phys Sportsmed 1988;16:143-51. [Crossref] [PubMed]
26. Alzahrani A, Hu S, Azorin-Peris V, et al. A multi-channel opto-electronic sensor to accurately monitor heart rate against motion artefact during exercise. Sensors (Basel) 2015;15:25681-702. [Crossref] [PubMed]
27. El-Amrawy F, Nounou MI. Are currently available wearable devices for activity tracking and heart rate monitoring accurate, precise, and medically beneficial? Healthc Inform Res 2015;21:315-20. [Crossref] [PubMed]
28. Bunn JA, Navalta JW, Fountaine CJ, et al. Current state of commercial wearable technology in physical activity monitoring 2015-2017. Int J Exerc Sci 2018;11:503-15. [PubMed]