Abstract
The intersection of global broadband technology and miniaturized high-capability computing devices has led to a revolution in the delivery of healthcare and the birth of telemedicine and mobile health (mHealth). Rapid advances in handheld imaging devices with other mHealth devices such as smartphone apps and wearable devices are making great strides in the field of cardiovascular imaging like never before. Although these technologies offer a bright promise in cardiovascular imaging, it is far from straightforward. The massive data influx from telemedicine and mHealth including cardiovascular imaging supersedes the existing capabilities of current healthcare system and statistical software. Artificial intelligence with machine learning is the one and only way to navigate through this complex maze of the data influx through various approaches. Deep learning techniques are further expanding their role by image recognition and automated measurements. Artificial intelligence provides limitless opportunity to rigorously analyze data. As we move forward, the futures of mHealth, telemedicine and artificial intelligence are increasingly becoming intertwined to give rise to precision medicine.
Introduction
Technological advancement has developed portable computer devices and miniaturized cardiac imaging devices. These devices with the simultaneous development of broadband technologies has led to a new frontier in communication by expanding the capabilities of information sharing among users worldwide. The effects of this digital landscape have permeated through multiple facets of daily life. Telemedicine and mobile health (mHealth), which is defined as use of mobile and wireless technologies to improve health care (1, 2), are becoming important in this digital landscape with cardiovascular medicine and the field of echocardiography being no exception. The Department of Health and Human Services estimates that more than 60% of all health care institutions in the United States currently use some form of telemedicine (3). Handheld imaging platforms and tele-interpreting has brought these trends into the field of echocardiography (4, 5).
Although big data generated by the telemedicine and cardiac imaging present great opportunity for cardiovascular research, this influx of data requires so much effort to integrate and interpret them that human cardiologists cannot digest all of it (6). Artificial intelligence (AI), including machine learning techniques, is increasingly recognized as a potential solution for facilitating a seamless transition between cardiologists and big data. AI can integrate the multifactorial information from many aspects of healthcare, including echocardiographic data, and can help cardiologists make better clinical decisions even in resource-limited areas where experts are not readily accessible. mHealth, telemedicine and AI offer bright promises intertwined with complex challenges in the field of cardiology and imaging. In this review, we will discuss the role of mHealth, telemedicine and AI in echocardiography.
Mobile health
Handheld imaging devices
As mobile computers and handheld imaging platforms become easily accessible and readily available, they present new paths of opportunity for the delivery and optimization of cardiovascular healthcare. Since the dawn of medicine, physical examination has been central to point-of-care diagnosis in cardiovascular medicine. The rapid rise of imaging devices which help physicians visualize the heart’s activities in real time have complemented physical examination and augmented clinical decision making. Despite the wide array of imaging capabilities at our disposal, correct diagnosis are not always made in time resulting in unfavorable outcomes (7). This has perpetuated a need, no a necessity for rapid and efficient diagnosis at bedside. The development of miniaturized handheld imaging platforms such as the pocket-size ultrasound can circumvent the obstacles of delayed diagnosis and reduce medical errors (7). There are several types of handheld ultrasounds with various capabilities; a laptop-based equipment has almost every 2D echocardiographic application, while a pocket-size ultrasound does not usually have full-scale color-flow and spectral Doppler capabilities (Fig. 1). The point-of-care ultrasound (POCUS) can fundamentally alter bedside medicine and be indispensable with physical examination. There are numerous studies which have clearly shown that POCUS is as efficient and effective compared to conventional machines. (8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18) Many researches have shown their capability in the assessment of valvular heart disease (19, 20), heart failure (21, 22, 23), coronary artery disease (22, 24) and so forth. (Table 1). Accuracy of POCUS has been reported well. For example, Abe et al. studied 130 patients with aortic stenosis and reported that pocket ultrasound was able to discriminate moderate-to-severe aortic stenosis with sensitivity 84% and specificity 90% even without quantitative Doppler information (19). Most recently, there are some smartphone-sized devices with image quality well enough for cardiac assessment using AI-based technologies. Some of these devices are supposedly financially cheap and help cardiologists in practice (e.g. Vscan, GE Healthcare and Butterfly IQ, Israel).
Type of handheld ultrasound machines. There are several types of handheld ultrasounds with various capabilities; a laptop-based equipment has almost every 2D echocardiographic application (panel A), while a pocket-size ultrasound does not usually have full-scale color-flow and spectral Doppler capabilities (panel B). Reproduced, with permission, from Chamsi-Pasha et al. (4).
Citation: Echo Research and Practice 6, 2; 10.1530/ERP-18-0081
Comparison of handheld ultrasound with reference standard.
| Study | Year | Number of subjects | Reference standard to PUS | Study findings |
|---|---|---|---|---|
| Prinz et al. (8) | 2011 | 349 | Standard echocardiography | Statistically significant agreement between PUS and high-end echocardiography (1.6 ± 0.5 vs 1.7 ± 0.4, P < 0.01), regional wall motion (κ = 0.73, P < 0.01), LV measurements (r = 0.99, P < 0.01), regurgitation detection (k = 0.9, P < 0.01) |
| Galderisi et al. (9) | 2010 | 304 | Standard echocardiography | The K between PUS and reference was 0.67 in the pooled population (0.84 by experts and 0.58 by trainees) |
| Testuz et al. (10) | 2013 | 104 | Standard echocardiography | Statistically significant agreement between PUS and reference for left ventricular function and pericardial effusion (kappa: 0.89 and 0.81). The agreement for aortic, mitral, tricuspid and left ventricular size was moderate (Kappa: 0.55–0.66) |
| Andersen et al. (11) | 2011 | 108 | Standard echocardiography | Strong agreement between PUS and reference for abdominal aorta and pericardial effusion was (r ≥ 0.92), right ventricular and valvular function (r ≥ 0.81). The correlation for aortic stenosis was (r = 0.62) |
| Skjetne et al. (7) | 2011 | 119 | Standard echocardiography | The PUS accurately assessed and diagnosed only 16% of patients in the cardiac unit. In 55% of patients, the reference had higher diagnostic value |
| Lafitte et al. (12) | 2011 | 100 | Standard echocardiography | The concordance between PUS and reference for LV function and morphology (κ = 0.91 and 0.96), left ventricular hypertrophy (k = 0.74), mitral regurgitation grades were 0.90, 0.95, and 1.00 |
| Michalski et al. (13) | 2012 | 220 | Standard echocardiography | There was excellent correlation between PUS and reference (r = 0.64–0.96, P < 0.001) |
| Biais et al. (14) | 2012 | 151 | Standard echocardiography | The PUS had good accordance with the reference in global left ventricular systolic dysfunction (κ = 0.87), pericardial effusion (κ = 0.75) |
| Prinz et al. (15) | 2012 | 320 | Standard echocardiography | In comparison to reference, substantial agreement in functional assessment (κ > 0.61, P < 0.01) and wall motion scoring (κ = 0.67, P < 0.01) could be observed over time. The correlation in left ventricular measurements (r > 0.98, P < 0.01) was very good |
| Fukuda et al. (16) | 2009 | 125 | Standard echocardiography | Left ventricular dimensions, fractional shortening, interventricular septum thickness, posterior wall thickness, left atrial dimension, and aortic diameter show excellent correlation (r = 0.87–0.98, all P < 0.001) |
| Mjolstad et al. (17) | 2012 | 196 | Standard echocardiography | Excellent agreement was observed between PUS and reference |
| Panoulas et al. (18) | 2013 | 122 | Standard echocardiography | After addition of PUS, there was improved diagnostic accuracy (Z = −7.761, P < 0.001) |
| Carlino et al. (25) | 2018 | 102 | Standard echocardiography | After addition of PUS, it helped improve diagnostic accuracy (all P < 0.01 vs single modalities) |
| Bhavnani et al. (39) | 2018 | 254 | Standard echocardiography | PUS had a shorter time to referral for intervention (83 ± 79 days vs 180 ± 101 days; P < 0.001). The PUS group had lower risk of hospitalization and death (15% vs 28%, adjusted hazard ratio: 0.41; P = 0.013) |
| Filipiak-Strzecka et al. (26) | 2017 | 100 | Standard echocardiography | There was statistically significant correlation between PUS and reference for intimal medial thickness (r = 0.58; 95% CI: 0.48–0.66; P < 0.0001) |
| Phillips et al. (22) | 2017 | 102 | Standard echocardiography | In relation to reference, PUS had values ranging from 85% for left atrial enlargement to 100% for cardiomegaly, but limited specificity of cardiomegaly at just 51% |
| Esposito et al. (27) | 2017 | 508 | Standard echocardiography | In a subgroup, PUS was compared with the standard for abdominal aorta size (rho = 0.966, P < 0.0001) |
| Cavallari et al. (28) | 2015 | 100 | Standard echocardiography | The PUS had a shorter time for examination (6.1 ± 1.2 min vs 13.1 ± 2.6 min, P < 0.0001) and saved waiting time (P < 0.001). No difference in conclusiveness between both groups (86 vs 96%; P = 0.08) |
| Khan et al. (29) | 2014 | 240 | Standard echocardiography | No discernable differences between both groups (P = 7.22 × 10(-7)). |
PUS, pocket-size ultrasound.
With powerful and affordable diagnostic imaging devices at the palm of our hands, POCUS can augment and add a significant impact on cardiovascular healthcare (25, 26, 27, 28, 29), especially in patients living in resource-limited areas.
Other mHealth devices
There are more than 160,000 health-related smartphone apps, such as apps for monitoring weight or diet control, available and these apps have been downloaded close to 660 million times (30, 31, 32). In addition, there have been many smartphone-connected devices and wearable devices available, which enable remote monitoring of health conditions including heart rhythm and blood pressure (30, 33, 34, 35). One of the hottest topics in the field is detection of atrial fibrillation using smartwatch. Tison et al. compared smartwatch data with standard ECG in 9750 patients for detecting atrial fibrillation (36). A deep learning-based algorithm showed excellent prediction of atrial fibrillation (C-statistic 0.97) with a sensitivity of 98% and specificity of 90.2%. Those apps and devices, along with other devices such as point-of-care measurements of B-type natriuretic peptide (BNP) (37) and implantable pulmonary artery pressure sensors (38), have potential to provide better identification of underlying diseases and improve their outcomes in communities (Fig. 2).
Interrogation of mHealth devices and use of artificial intelligence. Technological advancement has created a number of mobile health devices, which are available even in resource-limited areas. Involving remote experts using telemedicine helps appropriate diagnosis and management. Artificial intelligence can efficiently address the lack of experts and the influx of complex data generated by mHealth and telemedicine as well as advanced imaging modalities.
Citation: Echo Research and Practice 6, 2; 10.1530/ERP-18-0081
Integration of handheld imaging platforms with other mHealth devices
Bhavnani et al. conducted the first randomized trial of integration of POCUS with other mHealth devices in modern structural heart disease clinics in rural parts of India, under the ASE Foundation-Valvular Assessment Leading to Unexplored Echocardiographic Stratagems (ASEF-VALUES) (39). There were a total of 253 patients with structural heart disease randomized into two groups of mHealth clinic and standard healthcare. The main focus was the impact of mHealth with pocket-size echocardiography on medical decision making in patients with valvular heart disease in remote areas. The primary objective was time to referral for management for surgical or percutaneous intervention. The initial mHealth clinic was associated with shorter referral time for intervention (83 ± 79 days vs 180 ± 101 days; P < 0.001) and increased probability for intervention compared with standard healthcare (adjusted hazard ratio, 1.54; 95% CI, 0.96–2.47, P < 0.07). The patients assigned to mHealth clinic had lower hospitalization and death (15% vs 28%, adjusted hazard ratio, 0.41; 95% CI, 0.21–0.83; P < 0.013). In this study, the authors successfully integrated POCUS with other mHealth devices and showed that this integration can be associated with earlier medical interventions and favorable clinical outcome.
Telemedicine with POCUS and mHealth devices
Feasibility of POCUS in telemedicine
Thus, pocket-size ultrasound and other mHealth devices have allowed point-of-care screening of cardiovascular diseases to resource-limited communities. Furthermore, application of telemedicine technologies enables mHealth strategies even in remote areas with limited access to experts. Singh et al. (40) under American Society Echocardiography: Remote Echocardiography with Web-Based Assessments for Referral at a Distance (ASE-REWARD) performed a prospective study in order to test the feasibility of performing POCUS with long-distance Web-based assessment of recorded images. Using pocket-size ultrasound, 1023 studies were scanned in a rural region of India, and the images were sent to physicians in remote locations for review through Web-based platforms. The images were successfully uploaded and reviewed at a median time of 11:44 h. There was an excellent agreement in assessing valvular lesions, whereas the on-site readings were frequently modified by expert reviewers for left ventricular function and hypertrophy. The study successfully showed the feasibility of remote echocardiographic assessment and the incremental value of using Web-based remote assessment for facilitating appropriate mass triage of patients with suspected cardiac illnesses.
Choi et al. (41) tested the feasibility of remote interpretation of echocardiographic images on a smartphone. Eighty-nine patients underwent POCUS and the images were sent to remote experts who read them using smartphone apps. The authors found that 38% of on-site, non-expert diagnosis was revised by remote experts, whose interobserver agreement was excellent. The study suggested that remote interpretation is feasible and should be considered when POCUS is done by non-experts.
Limitations of POCUS
Although the benefits of POCUS are promising, there have been several challenges for its clinical application. One of the biggest concerns is the standardization of the quality of scan and interpretation (4). Because of its availability, POCUS can be used in more various situations and by wider range of observers than standard echocardiography. On the other hand, POCUS has limited ability in terms of image quality and applications such as pulse-wave Doppler. Scanning patients using POCUS and interpreting images by novice observers can result in overlooking important findings and wrong diagnosis (4). It is absolutely pivotal for all healthcare providers who use POCUS to be properly trained and understand the limitations of POUCS. Most professional societies require a minimum of 30 scans for basic training, but this number is not enough for accurate interpretation. Universal standardization of training is necessary for wide use of POCUS in clinical practice. Some of these limitations can be addressed by AI. For example, AI-based automated LVEF analysis program that works on PUS images (LVivo by DiA Imaging Analysis Ltd., Israel and Vscan by GE Healthcare) has been developed (42). This kind of programs will reduce the interobserver variability and help standardization of procedures. The lack of incentive for POCUS in US healthcare model is another problem, because more referrals and reimbursement for conventional echocardiography is more beneficial. A reward system is important to stimulate increased utilization.
Remote training and robot-assisted echocardiography
Even telemedicine enables remote assessment of acquired echocardiographic images, and appropriate acquisition itself requires expertise, which may limit its wider use in rural areas. Telemedicine also has a potential to address this issue through Web-based training. Bansal et al. (43) tested the feasibility of Web-based, real-time, hands-on, personalized training program of POCUS. Seventeen physicians in India were provided 6-h training of POCUS; nine had an on-site training and eight had an online training using transcontinental tele-echocardiography system. Although good-quality images were obtained more frequently by physicians trained on-site (90 vs 84%, P = 0.03), there were no difference between the two groups in agreement of the trained physicians’ diagnoses with expert interpretations. Such training, combined with Web-based integration of remote, expert interpretation of stored images, allows the delivery of echocardiographic expertise to remote communities, which could be of great help in optimizing cardiovascular health outcomes.
Robot-assisted remote echocardiography may be another solution. Boman et al. conducted randomized control trial and showed that real-time robot-assisted remote echocardiography followed by cardiologic consultation at a distance significantly reduced the total diagnostic process time (44).
Advanced echocardiography in contrast to POCUS
As discussed earlier, POCUS is promising and has a huge impact in expanding and complimenting physical examination. However, those handheld imaging platforms have limited function, and comprehensive and advanced echocardiography is definitely warranted in addition to POCUS. Recent advancement in echocardiography includes automation of measurements and analysis. Although speckle tracking echocardiography (STE) and 3D imaging have been the most promising methods in echocardiography for the past two decades, clinical use of these techniques is not sufficient due to time-consuming process. Automation of these techniques using AI algorithms are evolving and they help physicians and sonographers by reducing analysis time and increasing reproducibility (45, 46, 47). This is also a field where AI has a core role for evolution and for widespread use of the techniques. For example, although 3D echocardiography has been extensively reported to be superior to 2D echocardiography, the full adoption of this technique is not embraced due to time constraints and complicated measurement steps which disrupt clinical work flow. HeartModel (Philips Healthcare, Andover, MA, USA) is an AI-based fully automated quantification program for left heart chambers. The program dramatically reduces time for analysis (144 ± 32 to 26 ± 2 min, P < 0.0001) with even better interobserver measurement agreement compared with conventional 3D quantification (48, 49).
Artificial intelligence
Despite the transformative potential of mHealth and telemedicine, the data generated by these technologies are multifactorial and complex. In addition, imaging modalities including echocardiography also generate a huge amount of data; a single echo examination generates 2 gigabyte of information and annually there can be 15 petabytes of information produced (50). This large size of data would overwhelm current statistical software. AI is a field of computer science which mimics human thought process and learning capacity. AI could algorithmically quickly analyze and offer various interpretations of these elaborate datasets with lesser difficulty. With the rapid evolution of data, AI will be the primary and most efficient tool which brings the necessary revolution for integration of information into cardiovascular healthcare. In resource-limited situations where mHealth and telemedicine have an important role, well-trained AI may complement the lack of experts. AI techniques, such as machine learning and deep learning, unravel hidden patterns within heterogeneous datasets using a number of various algorithms (50). With the advent of AI, the paradigm is being fundamentally altered from current statistical tools to cardiovascular precision medicine (Fig. 3) (51).
Growth of publications in machine learning. The x- and y-axis shows the year and the number of publications in PubMed with ‘Cardiology’ and ‘Machine Learning’. The number of publications is rapidly growing, representing huge interest in the field. Reproduced, with permission, from Shameer et al. (50).
Citation: Echo Research and Practice 6, 2; 10.1530/ERP-18-0081
Type of machine learning
Machine learning is one subfield of AI, which aims at automatic discovery of regularities in data through the use of computer algorithms and generalizing those into new but similar data (Fig. 4). In general, machine learning tends to make less pre-assumption than traditional statistical method but requires greater data. Machine leaning techniques can be broadly split into supervised learning, unsupervised learning and reinforcement learning.
Association of artificial intelligence, machine learning and deep learning. Artificial intelligence (AI), though there are various definitions by itself, represents any techniques which enables computers mimic human behavior when it’s used in medical field. Machine learning is a subfield of AI, which aims at automatic discovery of regularities in data through the use of computer algorithms and generalizing those into new but similar data. Deep learning is a subset of machine learning, which makes the computation of multi-layer neural networks feasible.
Citation: Echo Research and Practice 6, 2; 10.1530/ERP-18-0081
In supervised learning, the database is labeled with outcome and classes. Supervised learning frequently groups an observation into one or more categories or outcomes (51). It is ultimately designed to show how the independent variable is linked to the dependent variable. A statistical model is generated from the data to create a model to predict an event or complication. Supervised learning proves to be very valuable in classifying phenotypically different patients (51). In contrast to supervised learning, unsupervised learning uses database with no prior label present (34). The purpose of unsupervised learning is to discover the relationship between variables. This machine-learning approach consists of clustering methods (hierarchical or K means), self-organizing maps, topological data analysis, information maximization analysis and finally deep learning. Reinforcement learning is derived from behavioral psychology. The algorithm learn and modify behavior through trial and error so as to maximize some notion of cumulative reward (51). Reinforcement learning is mostly used in game programs, such as AlphaGo software by DeepMind (52). Reinforcement learning has had limited role in healthcare so far.
Deep learning
Deep learning is a thriving discipline which learns complex hierarchical representations from data which has multiple levels of abstractions (53). It mimics the complexities of the human brain. Presently, deep learning is playing a prominent role in Facebook’s image recognition, speech recognition in Apple’s Siri and Amazon’s Alexa, Google brain and robots (53). Deep learning architecture utilizes an artificial neural network which contains multiple layers of neurons which facilitates reasoning and interpretation. Recent advances in graphic processing unit and cloud-based platforms have spurred the growth of deep learning.
Deep learning requires a large elaborate data sets which requires information sharing between institutions and organizations. If the dataset is not large enough, overfitting is an issue (50). It has multiple layers and performs analysis in a nonlinear manner. This also increases the training time. Assembling the neural network is also lengthy process. Powerful computing processing unit and cloud-based systems are often necessary for deep learning.
Comparison of machine learning with traditional statistics
Logistic regression is one of the most commonly used methods in statistics to predict outcomes (50). However, this technique requires a strong number of assumptions to help generate P values. Nevertheless, machine learning can be used in any data set without making any assumptions of the underlying data. Especially for classification, machine learning can be more accurate and predictive. Another difference is the capability to deal with complex data. Electronic health record contains a massive amount of information from billing, international disease classification, lab values, imaging and medications. This can exceed the capacity of logistic regression model. Other statistical approaches such as univariate significance screening or stepwise regression, but the results do not translate well for patient care. Complex interaction between variable may be difficult to analyze with traditional approaches. Churpek et al. showed how flexible algorithms in machine learning was superior to conventional logistic regression for clinical deterioration in wards in a large multihospital study (54). Popular risk scores such as Framingham risk score, CHADS2 and CHA2DS2-VASc score, and so forth were derived from large trials and registries (55). However, Cook et al. found that there was an overestimation of these pooled cohort equations, believed that big data analytics could resolve the issue (56).
The role of machine learning in cardiology and echocardiography
Many machine-learning and deep learning techniques can be applied to researches in echocardiography (Table 2) (38, 48, 57, 58, 59, 60). For example, we showed that supervised learning algorithm, including artificial neural networks, support vector machines and random forests, could differentiate athlete heart and hypertrophic cardiomyopathy using STE data more accurately than traditional measures (58). We have also used supervised learning approach with 15 STE variables and the four conventional echocardiographic variables and showed that machine learning was superior to other echo parameters for differentiating constrictive pericarditis from restrictive cardiomyopathy (57).
Examples of application of machine learning techniques to echocardiographic research.
| Study | Algorithm model | Brief algorithm description | Data source | Brief study description |
|---|---|---|---|---|
| Narula et al. (58) | (a) Support vector machine | Finds a gap in multidimensional data and classifies data based on gap | Echocardiographic data | To differentiate between athlete heart and hypertrophic cardiomyopathy |
| (b) Random forest | Decision tree-based method derived from creating a number of decision trees | |||
| (c) Artificial neural network | Learns in a manner similar to a biological network | |||
| Sengupta et al. (57) | Associative memory classifier-supervised learning | Used for making predictions based on a set of matrices. It is developed by observing co-occurrences of predictors from outcomes | Speckle tracking echocardiographic data | To differentiate between constrictive pericarditis and restrictive cardiomyopathy |
| Berikol et al. (48) | Artificial neural network | Echocardiographic data | Echocardiographic data and clinical factors used to stratify cardiovascular risk | |
| Lancaster et al. (59) | Hierarchical clustering | It classifies similar objects into the same groups called clusters by building a hierarchy based on the distance between patients | Echocardiographic data | To investigate the natural clustering of echocardiographic variables to measure left ventricular dysfunction and isolate high-risk phenotyping patterns |
| Abdolmanafi et al. (38) | Deep learning | It creates layered neural networks to extract and transform features and learn in supervised and/or unsupervised manners | Coronary optical coherence tomography images | To automatically classify coronary artery layers in coronary optical coherence tomography images in Kawasaki disease |
| Bai et al. (60) | Cardiac magnetic resonance | Deep learning was used to analyze short and long axis cardiac magnetic resonance imaging and compare with human performance | ||
Deep learning is being utilized for a number of image-based classifications. This machine learning approach is particularly useful for computer vision. Deep learning can track pattern recognition in cardiovascular imaging and heterogeneous syndromes. Left ventricular ejection fraction is usually assessed by manually tracing boundaries (53), but unfortunately this method can be subjective lack precision or reproducibility (61). Deep learning can greatly improve the accuracy of 2D STE and other imaging modalities (48, 59). This can be extended into other cardiovascular imaging modalities such as 3D STE and cardiac magnetic resonance imaging. It performs well even with noisy data such as strain imaging. Deep learning can be implemented into a number of cardiovascular diseases including heart failure, takotsubo cardiomyopathy, hypertension, atrial fibrillation, Brugada syndrome and so forth. It can categorize these conditions with new genotypes or phenotypes and innovative echocardiographic parameters can craft pathways for new therapies.
Recently, Zhang et al. developed a deep learning algorithm that enables fully automated interpretation of echocardiography (62). Using a huge (over 14,000) sample of echocardiographic studies, the algorithm achieved a 96% accuracy in image recognition for distinguishing between broad echocardiographic view classes (e.g. parasternal long axis from short axis), and 72–90% accuracy of image segmentation. Furthermore, the authors showed that the algorithm for automated quantification of cardiac structure and function was comparable or even superior to manual measurements across 11 internal consistency metrics (e.g. the correlation of left atrial and ventricular volume) and that the convolutional neural networks was successfully trained to detect hypertrophic cardiomyopathy, cardiac amyloidosis and pulmonary artery hypertension with high accuracy. Although the accuracy has not reached that of experts, application of deep learning to echocardiography interpretation is promising.
Future of artificial intelligence in cardiology
The rapid expansion of data is creating a moment of reckoning of sorts for cardiologists. With the development of POCUS integrated with mHealth devices and telemedicine, a concept which once seemed like a fantasy is now becoming a reality. AI, mainly machine learning techniques including deep learning, is the most effective means presently available to handle the sheer complexity data incoming from these evolutions. Compared to subspecialties of medicine, cardiologists have vast expanses of data at their disposal. As the complexities of data continue to grow, it is becoming imminent for an AI to be integrated into clinical practice. AI will become part and parcel of daily medicine, which is evidenced in the fields of radiology and pathology (63). It should be embraced not feared as it will enhance the clinical decision-making process. In the future, it may be necessary for all cardiologists to be physicians and data scientists simultaneously.
Conclusion
The burgeoning of mHealth, telemedicine and AI are the expanding the boundaries of echocardiography and cardiology. mHealth and telemedicine are establishing new bridges between patient and physician and helping underserved population to overcome previous barriers with their health care providers. AI is the truss support for these bridges. AI is the primary means and will be interconnected with the growth of these novel healthcare technologies for years to come. As mHealth and telemedicine create big data even in resource-limited areas where the number of experts is not sufficient and big data from these technologies are getting more and more complex, AI will assist cardiologists to provide more focused and personalized decision for the patients.
Declaration of interest
The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of this review.
Funding
This review did not receive any specific grant from any funding agency in the public, commercial or not-for-profit sector.
References
- 1↑
Burke LE, Ma J, Azar KM, Bennett GG, Peterson ED, Zheng Y, Riley W, Stephens J, Shah SH, Suffoletto B, et al. Current science on consumer use of mobile health for cardiovascular disease prevention: a scientific statement from the American Heart Association. Circulation 2015 132 1157–1213. (https://doi.org/10.1161/CIR.0000000000000232)
- 2↑
Tuckson RV, Edmunds M & Hodgkins ML. Telehealth. New England Journal of Medicine 2017 377 1585–1592. (https://doi.org/10.1056/NEJMsr1503323)
- 3↑
Office of Health Policy, Office of the Assistant Secretary for Planning and Evaulation. Report to Congress: E-health and telemedicine. Washington, DC, USA: Department of Health and Human Services, 2016. (available at: https://aspe.hhs.gov/sites/default/files/pdf/206751/TelemedicineE-HealthReport.pdf)
- 4↑
Chamsi-Pasha MA, Sengupta PP & Zoghbi WA. Handheld echocardiography: current state and future perspectives. Circulation 2017 136 2178–2188. (https://doi.org/10.1161/CIRCULATIONAHA.117.026622)
- 5↑
Johri AM, Durbin J, Newbigging J, Tanzola R, Chow R, De S & Tam J. Cardiac point-of-care ultrasound: state-of-the-art in Medical School education. Journal of the American Society of Echocardiography 2018 31 749–760. (https://doi.org/10.1016/j.echo.2018.01.014)
- 6↑
Konstam MA, Hill JA, Kovacs RJ, Harrington RA, Arrighi JA, Khera A & Academic Cardiology Section Leadership Council of the American College of Cardiology. The academic medical system: reinvention to survive the revolution in health care. Journal of the American College of Cardiology 2017 69 1305–1312. (https://doi.org/10.1016/j.jacc.2016.12.024)
- 7↑
Skjetne K, Graven T, Haugen BO, Salvesen Ø, Kleinau JO & Dalen H. Diagnostic influence of cardiovascular screening by pocket-size ultrasound in a cardiac unit. European Journal of Echocardiography 2011 12 737–743. (https://doi.org/10.1093/ejechocard/jer111)
- 8↑
Prinz C & Voigt JU. Diagnostic accuracy of a hand-held ultrasound scanner in routine patients referred for echocardiography. Journal of the American Society of Echocardiography 2011 24 111–116. (https://doi.org/10.1016/j.echo.2010.10.017)
- 9↑
Galderisi M, Santoro A, Versiero M, Lomoriello VS, Esposito R, Raia R, Farina F, Schiattarella PL, Bonito M, Olibet M, et al. Improved cardiovascular diagnostic accuracy by pocket size imaging device in non-cardiologic outpatients: the NaUSiCa (Naples ultrasound Stethoscope in Cardiology) study. Cardiovascular Ultrasound 2010 8 51. (https://doi.org/10.1186/1476-7120-8-51)
- 10↑
Testuz A, Muller H, Keller PF, Meyer P, Stampfli T, Sekoranja L, Vuille C & Burri H. Diagnostic accuracy of pocket-size handheld echocardiographs used by cardiologists in the acute care setting. European Heart Journal Cardiovascular Imaging 2013 14 38–42. (https://doi.org/10.1093/ehjci/jes085)
- 11↑
Andersen GN, Haugen BO, Graven T, Salvesen O, Mjolstad OC & Dalen H. Feasibility and reliability of point-of-care pocket-sized echocardiography. European Journal of Echocardiography 2011 12 665–670. (https://doi.org/10.1093/ejechocard/jer108)
- 12↑
Lafitte S, Alimazighi N, Reant P, Dijos M, Zaroui A, Mignot A, Lafitte M, Pillois X, Roudaut R & DeMaria A. Validation of the smallest pocket echoscopic device’s diagnostic capabilities in heart investigation. Ultrasound in Medicine and Biology 2011 37 798–804. (https://doi.org/10.1016/j.ultrasmedbio.2011.02.010)
- 13↑
Michalski B, Kasprzak JD, Szymczyk E & Lipiec P. Diagnostic utility and clinical usefulness of the pocket echocardiographic device. Echocardiography 2012 29 1–6. (https://doi.org/10.1111/j.1540-8175.2011.01553.x)
- 14↑
Biais M, Carrie C, Delaunay F, Morel N, Revel P & Janvier G. Evaluation of a new pocket echoscopic device for focused cardiac ultrasonography in an emergency setting. Critical Care 2012 16 R82. (https://doi.org/10.1186/cc11340)
- 15↑
Prinz C, Dohrmann J, van Buuren F, Bitter T, Bogunovic N, Horstkotte D & Faber L. Diagnostic performance of handheld echocardiography for the assessment of basic cardiac morphology and function: a validation study in routine cardiac patients. Echocardiography 2012 29 887–894. (https://doi.org/10.1111/j.1540-8175.2012.01728.x)
- 16↑
Fukuda S, Shimada K, Kawasaki T, Fujimoto H, Maeda K, Inanami H, Yoshida K, Jissho S, Taguchi H, Yoshiyama M, et al. Pocket-sized transthoracic echocardiography device for the measurement of cardiac chamber size and function. Circulation Journal 2009 73 1092–1096. (https://doi.org/10.1253/circj.CJ-08-1076)
- 17↑
Mjolstad OC, Dalen H, Graven T, Kleinau JO, Salvesen O & Haugen BO. Routinely adding ultrasound examinations by pocket-sized ultrasound devices improves inpatient diagnostics in a medical department. European Journal of Internal Medicine 2012 23 185–191. (https://doi.org/10.1016/j.ejim.2011.10.009)
- 18↑
Panoulas VF, Daigeler AL, Malaweera AS, Lota AS, Baskaran D, Rahman S & Nihoyannopoulos P. Pocket-size hand-held cardiac ultrasound as an adjunct to clinical examination in the hands of medical students and junior doctors. European Heart Journal Cardiovascular Imaging 2013 14 323–330. (https://doi.org/10.1093/ehjci/jes140)
- 19↑
Abe Y, Ito M, Tanaka C, Ito K, Naruko T, Itoh A, Haze K, Muro T, Yoshiyama M & Yoshikawa J. A novel and simple method using pocket-sized echocardiography to screen for aortic stenosis. Journal of the American Society of Echocardiography 2013 26 589–596. (https://doi.org/10.1016/j.echo.2013.03.008)
- 20↑
Furukawa A, Abe Y, Ito M, Tanaka C, Ito K, Komatsu R, Haze K, Naruko T, Yoshiyama M & Yoshikawa J. Prediction of aortic stenosis-related events in patients with systolic ejection murmur using pocket-sized echocardiography. Journal of Cardiology 2017 69 189–194. (https://doi.org/10.1016/j.jjcc.2016.02.021)
- 21↑
Gustafsson M, Alehagen U & Johansson P. Imaging congestion with a pocket ultrasound device: prognostic implications in patients With chronic heart failure. Journal of Cardiac Failure 2015 21 548–554. (https://doi.org/10.1016/j.cardfail.2015.02.004)
- 22↑
Phillips CT & Manning WJ. Advantages and pitfalls of pocket ultrasound vs daily chest radiography in the coronary care unit: a single-user experience. Echocardiography 2017 34 656–661. (https://doi.org/10.1111/echo.13509)
- 23↑
Russell FM, Ehrman RR, Cosby K, Ansari A, Tseeng S, Christain E & Bailitz J. Diagnosing acute heart failure in patients with undifferentiated dyspnea: a lung and cardiac ultrasound (LuCUS) protocol. Academic Emergency Medicine 2015 22 182–191. (https://doi.org/10.1111/acem.12570)
- 24↑
Nishigami K. Point-of-care echocardiography for aortic dissection, pulmonary embolism and acute coronary syndrome in patients with killer chest pain: EASY screening focused on the assessment of effusion, aorta, ventricular size and shape and ventricular asynergy. Journal of Echocardiography 2015 13 141–144. (https://doi.org/10.1007/s12574-015-0265-1)
- 25↑
Carlino MV, Paladino F, Sforza A, Serra C, Liccardi F, de Simone G & Mancusi C. Assessment of left atrial size in addition to focused cardiopulmonary ultrasound improves diagnostic accuracy of acute heart failure in the emergency department. Echocardiography 2018 35 785–791. (https://doi.org/10.1111/echo.13851)
- 26↑
Filipiak-Strzecka D, Kasprzak JD, Szymczyk E, Wejner-Mik P & Lipiec P. Bedside screening with the use of pocket-size imaging device can be useful for ruling out carotid artery stenosis in patients scheduled for cardiac surgery. Echocardiography 2017 34 716–722. (https://doi.org/10.1111/echo.13507)
- 27↑
Esposito R, Ilardi F, Schiano Lomoriello V, Sorrentino R, Sellitto V, Giugliano G, Esposito G, Trimarco B & Galderisi M. Identification of the main determinants of abdominal aorta size: a screening by pocket size imaging device. Cardiovascular Ultrasound 2017 15 2. (https://doi.org/10.1186/s12947-016-0094-z)
- 28↑
Cavallari I, Mega S, Goffredo C, Patti G, Chello M & Di Sciascio G. Hand-held echocardiography in the setting of pre-operative cardiac evaluation of patients undergoing non-cardiac surgery: results from a randomized pilot study. International Journal of Cardiovascular Imaging 2015 31 995–1000. (https://doi.org/10.1007/s10554-015-0656-4)
- 29↑
Khan HA, Wineinger NE, Uddin PQ, Mehta HS, Rubenson DS & Topol EJ. Can hospital rounds with pocket ultrasound by cardiologists reduce standard echocardiography? American Journal of Medicine 2014 127 669.e1–669.e7. (https://doi.org/10.1016/j.amjmed.2014.03.015)
- 30↑
Bhavnani SP, Narula J & Sengupta PP. Mobile technology and the digitization of healthcare. European Heart Journal 2016 37 1428–1438. (https://doi.org/10.1093/eurheartj/ehv770)
- 31↑
Chow CK, Ariyarathna N, Islam SM, Thiagalingam A & Redfern J. mHealth in cardiovascular health care. Heart, Lung and Circulation 2016 25 802–807. (https://doi.org/10.1016/j.hlc.2016.04.009)
- 32↑
Eapen ZJ, Turakhia MP, McConnell MV, Graham G, Dunn P, Tiner C, Rich C, Harrington RA, Peterson ED & Wayte P. Defining a mobile health roadmap for cardiovascular health and disease. Journal of the American Heart Association 2016 5 e003119. (https://doi.org/10.1161/JAHA.115.003119)
- 33↑
Tarakji KG, Wazni OM, Callahan T, Kanj M, Hakim AH, Wolski K, Wilkoff BL, Saliba W & Lindsay BD. Using a novel wireless system for monitoring patients after the atrial fibrillation ablation procedure: the iTransmit study. Heart Rhythm 2015 12 554–559. (https://doi.org/10.1016/j.hrthm.2014.11.015)
- 34↑
Williams B, Lacy PS, Baschiera F, Brunel P & Dusing R. Novel description of the 24-hour circadian rhythms of brachial versus central aortic blood pressure and the impact of blood pressure treatment in a randomized controlled clinical trial: the Ambulatory Central Aortic Pressure (AmCAP) Study. Hypertension 2013 61 1168–1176. (https://doi.org/10.1161/HYPERTENSIONAHA.111.00763)
- 35↑
Brooks GC, Vittinghoff E, Iyer S, Tandon D, Kuhar P, Madsen KA, Marcus GM, Pletcher MJ & Olgin JE. Accuracy and usability of a self-administered 6-minute walk test smartphone application. Circulation: Heart Failure 2015 8 905–913. (https://doi.org/10.1161/CIRCHEARTFAILURE.115.002062)
- 36↑
Tison GH, Sanchez JM, Ballinger B, Singh A, Olgin JE, Pletcher MJ, Vittinghoff E, Lee ES, Fan SM, Gladstone RA, et al. Passive detection of atrial fibrillation using a commercially available smartwatch. JAMA Cardiology 2018 3 409–416. (https://doi.org/10.1001/jamacardio.2018.0136)
- 37↑
Maisel A, Barnard D, Jaski B, Frivold G, Marais J, Azer M, Miyamoto MI, Lombardo D, Kelsay D, Borden K, et al. Primary results of the HABIT trial (heart failure assessment with BNP in the home). Journal of the American College of Cardiology 2013 61 1726–1735. (https://doi.org/10.1016/j.jacc.2013.01.052)
- 38↑
Abdolmanafi A, Duong L, Dahdah N & Cheriet F. Deep feature learning for automatic tissue classification of coronary artery using optical coherence tomography. Biomedical Optics Express 2017 8 1203–1220. (https://doi.org/10.1364/BOE.8.001203)
- 39↑
Bhavnani SP, Sola S, Adams D, Venkateshvaran A, Dash PK, Sengupta PP & ASEF-VALUES Investigators. A randomized trial of pocket-echocardiography integrated mobile health device assessments in modern structural heart disease clinics. JACC: Cardiovascular Imaging 2018 11 546–557. (https://doi.org/10.1016/j.jcmg.2017.06.019)
- 40↑
Singh S, Bansal M, Maheshwari P, Adams D, Sengupta SP, Price R, Dantin L, Smith M, Kasliwal RR, Pellikka PA, et al. American Society of Echocardiography: remote echocardiography with web-based assessments for referrals at a distance (ASE-REWARD) study. Journal of the American Society of Echocardiography 2013 26 221–233. (https://doi.org/10.1016/j.echo.2012.12.012)
- 41↑
Choi BG, Mukherjee M, Dala P, Young HA, Tracy CM, Katz RJ & Lewis JF. Interpretation of remotely downloaded pocket-size cardiac ultrasound images on a web-enabled smartphone: validation against workstation evaluation. Journal of the American Society of Echocardiography 2011 24 1325–1330. (https://doi.org/10.1016/j.echo.2011.08.007)
- 42↑
GE Healthcare. Vscan Extend datasheet. Chicago, IL, USA: General Electric Company, 2018. (available at: https://www.gehealthcare.com/-/media/fdbbc3f456914f5dbc3cc44cb866ffb5.pdf)
- 43↑
Bansal M, Singh S, Maheshwari P, Adams D, McCulloch ML, Dada T, Sengupta SP, Kasliwal RR, Pellikka PA, Sengupta PP, et al. Value of interactive scanning for improving the outcome of new-learners in transcontinental tele-echocardiography (VISION-in-tele-Echo) study. Journal of the American Society of Echocardiography 2015 28 75–87. (https://doi.org/10.1016/j.echo.2014.09.001)
- 44↑
Boman K, Olofsson M, Berggren P, Sengupta PP & Narula J. Robot-assisted remote echocardiographic examination and teleconsultation: a randomized comparison of time to diagnosis with standard of care referral approach. JACC: Cardiovascular Imaging 2014 7 799–803. (https://doi.org/10.1016/j.jcmg.2014.05.006)
- 45↑
Kagiyama N, Toki M, Hara M, Fukuda S, Aritaka S, Miki T, Ohara M, Hayashida A, Hirohata A, Yamamoto K, et al. Efficacy and accuracy of novel automated mitral valve quantification: three-dimensional transesophageal echocardiographic study. Echocardiography 2016 33 756–763. (https://doi.org/10.1111/echo.13135)
- 46↑
Medvedofsky D, Mor-Avi V, Byku I, Singh A, Weinert L, Yamat M, Kruse E, Ciszek B, Nelson A, Otani K, et al. Three-dimensional echocardiographic automated quantification of left heart chamber volumes using an adaptive analytics algorithm: feasibility and impact of image quality in nonselected patients. Journal of the American Society of Echocardiography 2017 30 879–885. (https://doi.org/10.1016/j.echo.2017.05.018)
- 47↑
Mor-Avi V, Lang RM, Badano LP, Belohlavek M, Cardim NM, Derumeaux G, Galderisi M, Marwick T, Nagueh SF, Sengupta PP, et al. Current and evolving echocardiographic techniques for the quantitative evaluation of cardiac mechanics: ASE/EAE consensus statement on methodology and indications endorsed by the Japanese Society of Echocardiography. Journal of the American Society of Echocardiography 2011 24 277–313. (https://doi.org/10.1016/j.echo.2011.01.015)
- 48↑
Tsang W, Salgo IS, Medvedofsky D, Takeuchi M, Prater D, Weinert L, Yamat M, Mor-Avi V, Patel AR & Lang RM. Transthoracic 3D echocardiographic left heart chamber quantification using an automated adaptive analytics algorithm. JACC: Cardiovascular Imaging 2016 9 769–782. (https://doi.org/10.1016/j.jcmg.2015.12.020)
- 49↑
Medvedofsky D, Mor-Avi V, Amzulescu M, Fernandez-Golfin C, Hinojar R, Monaghan MJ, Otani K, Reiken J, Takeuchi M, Tsang W, et al. Three-dimensional echocardiographic quantification of the left-heart chambers using an automated adaptive analytics algorithm: multicentre validation study. European Heart Journal Cardiovascular Imaging 2018 19 47–58. (https://doi.org/10.1093/ehjci/jew328)
- 50↑
Shameer K, Johnson KW, Glicksberg BS, Dudley JT & Sengupta PP. Machine learning in cardiovascular medicine: are we there yet? Heart 2018 104 1156–1164. (https://doi.org/10.1136/heartjnl-2017-311198)
- 51↑
Johnson KW, Torres Soto J, Glicksberg BS, Shameer K, Miotto R, Ali M, Ashley E & Dudley JT. Artificial intelligence in cardiology. Journal of the American College of Cardiology 2018 71 2668–2679. (https://doi.org/10.1016/j.jacc.2018.03.521)
- 52↑
DeepMind. AlphaGo. London, UK: DeepMind Technologies Limited, 2019. (available at: https://deepmind.com/research/alphago/)
- 53↑
Krittanawong C, Zhang H, Wang Z, Aydar M & Kitai T. Artificial intelligence in precision cardiovascular medicine. Journal of the American College of Cardiology 2017 69 2657–2664. (https://doi.org/10.1016/j.jacc.2017.03.571)
- 54↑
Churpek MM, Yuen TC, Winslow C, Meltzer DO, Kattan MW & Edelson DP. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Critical Care Medicine 2016 44 368–374. (https://doi.org/10.1097/CCM.0000000000001571)
- 55↑
Goff DC Jr, Lloyd-Jones DM, Bennett G, Coady S, D’Agostino RB Sr, Gibbons R, Greenland P, Lackland DT, Levy D, O’Donnell CJ, et al.2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Journal of the American College of Cardiology 2014 63 2935–2959. (https://doi.org/10.1016/j.jacc.2013.11.005)
- 56↑
Cook NR & Ridker PM. Calibration of the pooled cohort equations for atherosclerotic cardiovascular disease: an update. Annals of Internal Medicine 2016 165 786–794. (https://doi.org/10.7326/M16-1739)
- 57↑
Sengupta PP, Huang YM, Bansal M, Ashrafi A, Fisher M, Shameer K, Gall W & Dudley JT. Cognitive machine-learning algorithm for cardiac imaging: a pilot study for differentiating constrictive pericarditis from restrictive cardiomyopathy. Circulation: Cardiovascular Imaging 2016 9 e004330. (https://doi.org/10.1161/CIRCIMAGING.115.004330)
- 58↑
Narula S, Shameer K, Salem Omar AM, Dudley JT & Sengupta PP. Machine-learning algorithms to automate morphological and functional assessments in 2D echocardiography. Journal of the American College of Cardiology 2016 68 2287–2295. (https://doi.org/10.1016/j.jacc.2016.08.062)
- 59↑
Lancaster MC, Salem Omar AM, Narula S, Kulkarni H, Narula J & Sengupta PP. Phenotypic clustering of left ventricular diastolic function parameters: patterns and prognostic relevance. JACC: Cardiovascular Imaging 2018 [epub]. (https://doi.org/10.1016/j.jcmg.2018.02.005)
- 60↑
Bai W, Sinclair M, Tarroni G, Oktay O, Rajchl M, Vaillant G, Lee AM, Aung N, Lukaschuk E, Sanghvi MM, et al. Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. Journal of Cardiovascular Magnetic Resonance 2018 20 65. (https://doi.org/10.1186/s12968-018-0471-x)
- 61↑
Pellikka PA, She L, Holly TA, Lin G, Varadarajan P, Pai RG, Bonow RO, Pohost GM, Panza JA, Berman DS, et al. Variability in ejection fraction measured by echocardiography, gated single-photon emission computed tomography, and cardiac magnetic resonance in patients with coronary artery disease and left ventricular dysfunction. JAMA Network Open 2018 1 e181456. (https://doi.org/10.1001/jamanetworkopen.2018.1456)
- 62↑
Zhang J, Gajjala S, Agrawal P, Tison GH, Hallock LA, Beussink-Nelson L, Lassen MH, Fan E, Aras MA, Jordan C, et al. Fully automated echocardiogram interpretation in clinical practice. Circulation 2018 138 1623–1635. (https://doi.org/10.1161/CIRCULATIONAHA.118.034338)
- 63↑
Al’Aref SJ, Anchouche K, Singh G, Slomka PJ, Kolli KK, Kumar A, Pandey M, Maliakal G, van Rosendael AR, Beecy AN, et al. Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging. European Heart Journal 2018. (https://doi.org/10.1093/eurheartj/ehy404)
