Advertisement

Machine learning can support dispatchers to better and faster recognize out-of-hospital cardiac arrest during emergency calls: A retrospective study

Open AccessPublished:March 05, 2021DOI:https://doi.org/10.1016/j.resuscitation.2021.02.041

      Abstract

      Aim

      Fast recognition of out-of-hospital cardiac arrest (OHCA) by dispatchers might increase survival. The aim of this observational study of emergency calls was to (1) examine whether a machine learning framework (ML) can increase the proportion of calls recognizing OHCA within the first minute compared with dispatchers, (2) present the performance of ML with different false positive rate (FPR) settings, (3) examine call characteristics influencing OHCA recognition.

      Methods

      ML can be configured with different FPR settings, i.e., more or less inclined to suspect an OHCA depending on the predefined setting. ML OHCA recognition within the first minute is evaluated with a 1.5 FPR as the primary endpoint, and other FPR settings as secondary endpoints. ML was exposed to a random sample of emergency calls from 2018. Voice logs were manually audited to evaluate dispatchers time to recognition.

      Results

      Of 851 OHCA calls, the ML recognized 36% (n = 305) within 1 min compared with 25% (n = 213) by dispatchers. The recognition rate at any time during the call was 86% for ML and 84% for dispatchers, with a median time to recognition of 72 versus 94 s. OHCA recognized by both ML and dispatcher showed a 28 s mean difference in favour of ML (P < 0.001). ML with higher FPR settings reduced recognition times.

      Conclusion

      ML recognized a higher proportion of OHCA within the first minute compared with dispatchers and has the potential to be a supportive tool during emergency calls. The optimal FPR settings need to be evaluated in a prospective study.

      Keywords

      Introduction

      The mean incidence of out of hospital cardiac arrest (OHCA) is 84 per 100,000 population per year,
      • Gräsner J.T.
      • Lefering R.
      • Koster R.W.
      EuReCa ONE-27 Nations, ONE Europe, ONE Registry: a prospective one month analysis of out-of-hospital cardiac arrest outcomes in 27 countries in Europe.
      and the corresponding incidence rate of resuscitation is 56 per 100,000 population. The overall survival rate is 8%.
      • Gräsner J.T.
      • Wnent J.
      • Herlitz J.
      • et al.
      Survival after out-of-hospital cardiac arrest in Europe - results of the EuReCa TWO study.
      In Sweden, the incidence of OHCA is 60 per 100,000 population, and the incidence of resuscitation by bystanders and first responders is 45 per 100,000 population.
      • Herlitz J.
      Hjärt- och lungräddningsregistrets årsrapport.
      Cardiopulmonary resuscitation (CPR) and use of an automated external defibrillator (AED) by lay persons and first responders can increase survival in 7 of 10 cases.
      • Herlitz J.
      Hjärt- och lungräddningsregistrets årsrapport.
      • Perkins G.D.
      • Handley A.J.
      • Koster R.W.
      • et al.
      European Resuscitation Council Guidelines for Resuscitation 2015. Section 2. Adult basic life support and automated external defibrillation.
      • Hollenberg J.
      • Riva G.
      • Bohm K.
      • et al.
      Dual dispatch early defibrillation in out-of-hospital cardiac arrest: the SALSA-pilot.
      • Hasselqvist-Ax I.
      • Nordberg P.
      • Herlitz J.
      • et al.
      Dispatch of firefighters and police officers in out-of-hospital cardiac arrest: a nationwide prospective cohort trial using propensity score analysis.
      • Ringh M.
      • Jonsson M.
      • Nordberg P.
      • et al.
      Survival after public access defibrillation in Stockholm, Sweden — a striking success.
      Recognition of OHCA by dispatchers is time critical; the earlier recognition, the earlier start of telephone assisted CPR (T-CPR) can be initiated in parallel with dispatch of emergency medical services (EMS).
      • Perkins G.D.
      • Handley A.J.
      • Koster R.W.
      • et al.
      European Resuscitation Council Guidelines for Resuscitation 2015. Section 2. Adult basic life support and automated external defibrillation.
      • Drennan I.R.
      • Geri G.
      • Brooks S.
      • Couper K.
      • et al.
      Diagnosis of out-of-hospital cardiac arrest by emergency medical dispatch: a diagnostic systematic review.
      Shortening the time to CPR and EMS response time is crucial to increase survival in OHCA.
      • Bürger A.
      • Wnent J.
      • Bohn A.
      • et al.
      The effect of ambulance response time on survival following out-of-hospital cardiac arrest.
      • Hara M.
      • Hayashi K.
      • Hikoso S.
      • Sakata Y.
      • Kitamura T.
      Different impacts of time from collapse to first cardiopulmonary resuscitation on outcomes after witnessed out-of-hospital cardiac arrest in adults, circulation: cardiovascular quality and outcomes.
      • Holmén J.
      • Herlitz J.
      • Ricksten S.E.
      • et al.
      Shortening ambulance response time increases survival in out-of-hospital cardiac arrest.
      • Rajan S.
      • Wissenberg M.
      • Folke F.
      • et al.
      Association of bystander cardiopulmonary resuscitation and survival according to ambulance response times after out-of-hospital cardiac arrest.
      Recognition of OHCA among all incoming calls is a challenging task, approximately only 1% of all emergency calls involve an OHCA.
      • Viereck S.
      • Møller T.P.
      • Ersbøll A.K.
      • et al.
      Recognising out-of-hospital cardiac arrest during emergency calls increases bystander cardiopulmonary resuscitation and survival.
      Furthermore, witnessed OHCA and patients presenting with agonal breathing and/or seizures have been shown to introduce complexity to the recognition process.
      • Viereck S.
      • Møller T.P.
      • Ersbøll A.K.
      • et al.
      Recognising out-of-hospital cardiac arrest during emergency calls increases bystander cardiopulmonary resuscitation and survival.
      • Viereck S.
      • Møller T.P.
      • Rothman J.P.
      • Folke F.
      • Lippert F.K.
      Recognition of out-of-hospital cardiac arrest during emergency calls — a systematic review of observational studies.
      • Møller T.P.
      • Andréll C.
      • Viereck S.
      • Todorova L.
      • Friberg H.
      • Lippert F.K.
      Recognition of out-of-hospital cardiac arrest by medical dispatchers in emergency medical dispatch centres in two countries.
      • Schwarzkoph M.
      • Yin L.
      • Hergert L.
      • Drucker C.
      • Counts C.R.
      • Eisenberg M.
      Seizure-like presentation in OHCA creates barriers to dispatch recognition of cardiac arrest.
      • Breckwoldt J.
      • Schloesser S.
      • Arntz H.-R.
      Perceptions of collapse and assessment of cardiac arrest by bystanders of out-of-hospital cardiac arrest (OOHCA).
      Delay in recognition and the proportion of correct recognition vary greatly. In a review of 16 observational studies, the median sensitivity for recognition of OHCA was 74% (range, 14%–97%).
      • Viereck S.
      • Møller T.P.
      • Rothman J.P.
      • Folke F.
      • Lippert F.K.
      Recognition of out-of-hospital cardiac arrest during emergency calls — a systematic review of observational studies.
      A multicentre study including Emergency Medical Dispatch Centres (EMDC) from Copenhagen, Oslo and Stockholm showed 77% OHCA recognition in Copenhagen, 79% in Stockholm and 96% in Oslo.
      • Hardeland C.
      • Claesson A.
      • Blomberg S.N.F.
      • et al.
      Differences in out-of-hospital cardiac arrest emergency dispatch recognition and dispatcher assisted CPR instructions — a Scandinavian multicentre study.
      The American Heart Association (AHA) has set a “golden criterion standard” for EMDC, aiming at recognizing OHCA in a median time of 60 s.
      • Kurz M.C.
      • Bobrow B.J.
      • Buckingham J.
      • et al.
      Telecommunicator cardiopulmonary resuscitation: a policy statement from the American Heart Association.
      There is an urgent need for new and novel methods that can shorten the time for OHCA recognition and start of treatment. Machine learning (ML) with artificial intelligence can be used to assist with clinical decisions.
      • De Fauw J.
      • Ledsam J.R.
      • Romera-Paredes B.
      • et al.
      Clinically applicable deep learning for diagnosis and referral in retinal disease.
      • Gulshan V.
      • Peng L.
      • Coram M.
      • et al.
      Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.
      • Ceccarelli F.
      • Sciandrone M.
      • Perricone C.
      • et al.
      Prediction of chronic damage in systemic lupus erythematosus by using machine-learning models.
      • Esteva A.
      • Kuprel B.
      • Novoa R.A.
      • et al.
      Dermatologist-level classification of skin cancer with deep neural networks.
      A recent study demonstrated that ML has the potential to recognize OHCA during emergency calls with higher sensitivity and specificity compared with dispatchers.
      • Blomberg S.N.
      • Folke F.
      • Ersbøll A.K.
      • et al.
      Machine learning as a supportive tool to recognize cardiac arrest in emergency calls.
      The performance of the ML in recognizing OHCAs in emergency calls is dependent on its predefined false positive rate (FPR) setting.
      The primary aim of this observational study of Swedish emergency calls is to examine whether an ML framework can increase the proportion of recognized OHCA calls within the first minute compared with dispatchers. The secondary aims are to present ML performance with different FPR settings and to examine the call characteristics influencing OHCA recognition.

      Methods

      In this retrospective, registry-based study, ML was used to assist in recognizing OHCA in voice logs of Swedish emergency calls from 2018.

      Machine learning framework

      Machine learning is a way of programming computer software. Instead of programming instructions about what action to be executed, the computer utilizes complex statistical models to learn from large amounts of data to then understand problems and solve them.
      • Kwon J.
      • Lee Y.
      • Lee Y.
      • Lee S.
      • Park J.
      An algorithm based on deep learning for predicting in‐hospital cardiac arrest.
      • Stewart J.
      • Sprivulis P.
      • Dwivedi G.
      Artificial intelligence and machine learning in emergency medicine.
      The ML based on supervised learning used in this study was developed by Corti ApS using artificial intelligence technology.
      AI for Patient Consultations.
      The framework contains two ML models: an automatic speech recognition (ASR) model transcribes speech to text and an OHCA detection model predicts OHCA events from transcribed speech in real time.

      Automatic speech recognition

      ASR is a deep neural network using a model based on Connectionist Temporal Classification.
      • Borgholt L.
      • Havtorn J.D.
      • Agić Ž
      • Søgaard A.
      • Maaløe L.
      • Igel C.
      Do end-to-end speech recognition models care about context?.
      To train the ASR model in the Swedish language, a total of 45 h of uniform random selected Swedish emergency calls from 2015 concerning all types of emergencies were manually transcribed into written text. All text files were then used as labels for the ASR model to be trained in understanding the Swedish language. After the training phase, a validation and tuning phase was conducted.

      OHCA detection model

      The OHCA detection model is a densely connected deep neural network model to detect an OHCA in the transcribed text produced by the ASR model. For each second of raw audio, the ML predicts whether there is an OHCA from the accumulated audio sequence. In training the OHCA detection model, we used 3944 calls labelled as OHCA reported to the Swedish Registry of Cardiopulmonary Resuscitation (SRCR) by the EMS during 2016 and 39,888 calls labelled as no OHCA.
      The ML can be configured with different FPR settings, meaning the ML can be more or less inclined to suspect an OHCA depending on its predefined setting. This study has a predefined 1.5% FPR setting in the primary outcome analysis. Different FPR settings and their impact on the outcome were also analysed.

      Dispatchers

      When dialling the Swedish national emergency number, the call is received by a dispatcher at one of the 15 EMDCs run by SOS Alarm AB. In Sweden, the dispatchers at EMDCs handle emergency calls concerning a medical emergency from start to finish, except in three regions where the call is redirected to local EMDCs run by local EMS organizations. Dispatchers at SOS Alarm undergo a general training program for 14 weeks before entering an answering position taking emergency calls. Annually, all dispatchers must be certified to continue answering emergency calls. Dispatchers and registered nurses at SOS Alarm are supported in their medical decision making by a criteria-based medical index.

      Swedish Registry of Cardiopulmonary Resuscitation

      EMS in Sweden report OHCAs to the SRCR when bystanders or EMS have initiated resuscitation efforts. The SRCR adheres to Utstein template of OHCA registration.
      • Jacobs I.
      • Nadkarni V.
      • Bahr J.
      • et al.
      Cardiac arrest and cardiopulmonary resuscitation outcome reports: update and simplification of the Utstein templates for resuscitation registries. A statement for healthcare professionals from a task force of the International Liaison Committee on Resuscitation (American Heart Association, European Resuscitation Council, Australian Resuscitation Council, New Zealand Resuscitation Council.
      The SRCR is a national quality register and has been described elsewhere.
      • Strömsöe A.
      • Svensson L.
      • Axelsson ÅB.
      • Göransson K.
      • Todorova L.
      • Herlitz J.
      Validity of reported data in the Swedish Cardiac Arrest Register in selected parts in Sweden.
      • Emilsson L.
      • Lindahl B.
      • Köster M.
      • Lambe M.
      • Ludvigsson J.F.
      Review of 103 Swedish healthcare quality registries.

      Study population

      The ML was tested on two uniform random cohorts to evaluate its sensitivity and specificity: (a) validated OHCA calls, and (b) priority 1–4 calls (no OHCA).

      OHCA calls: cohort A

      In 2018, a total of 6135 cases of OHCA were registered by EMS personnel to the SRCR. After excluding OHCA calls witnessed by the EMS personnel, calls not available at the time of extraction and calls handled by care providers from regions where SOS Alarm does not assess the medical urgency and apply a dispatch code, a random sample were extracted for inclusion. In addition, in a subgroup analysis, we excluded calls where CPR was in progress (Fig. 1).
      Fig. 1
      Fig. 1Flowchart of the study: OHCA calls (cohort A).

      Priority 1–4 calls: cohort B

      In 2018, 1,140,434 emergency calls concerning health care were answered by EMDCs in Sweden. Cohort B consisted of a random sample (n = 85,205) of priority 1–4 calls, with the following calls excluded: validated OHCA registered in the SRCR, calls logged with a dispatch code or manual note that could possibly involve an OHCA, lifesavers activated through mobile text message, calls from regions with local EMDCs, and calls that had more than one case folder or with more than one emergency call to minimize the risk of duplicates in the sample (Supplementary Fig. 1).

      Manual audit

      To evaluate dispatchers’ performance of OHCA, a manual audit of all voice logs in cohort A was performed by two investigators. A modified version of the Cardiac Arrest Registry to Enhance Survival (CARES) dispatcher-assisted CPR data dictionary was used to extract data from the voice logs.

      CARES Dispather-Assisted CPR Data Dictionary. n.d. (Accessed 26 November 2020, at https://mycares.net/sitepages/uploads/2015/CARES%20Dispatcher-Assisted%20CPR%20Data%20Dictionary%2012-6-15.pdf).

      Dispatchers’ recognition of an OHCA was defined as when dispatchers mentioned the presence of an OHCA or the need for CPR. Time to recognition was measured from the time the dispatcher answered the emergency call until the definition of OHCA recognition was achieved.
      • Blomberg S.N.
      • Folke F.
      • Ersbøll A.K.
      • et al.
      Machine learning as a supportive tool to recognize cardiac arrest in emergency calls.

      CARES Dispather-Assisted CPR Data Dictionary. n.d. (Accessed 26 November 2020, at https://mycares.net/sitepages/uploads/2015/CARES%20Dispatcher-Assisted%20CPR%20Data%20Dictionary%2012-6-15.pdf).

      In addition, a random sample of OHCAs suspected by the ML (false positives) in cohort B were manually audited to evaluate if there could be unregistered OHCA calls present.

      Outcome measures

      The primary endpoint was the proportion (%) of OHCAs recognized within 60 s, ML versus dispatcher. Secondary endpoints were the proportion (%) of OHCAs recognized within 90 s, median time to recognition, recognition rate at any timepoint during the call, time difference between OHCA recognized by both ML and dispatcher (paired observation), test of different FPR settings and characteristics influencing OHCA recognition.

      Statistical analysis

      Data are presented as the proportion (%), median (interquartile range) or mean (standard deviation). For paired observations, the mean differences in time to recognition between groups were analysed with the Wilcoxon signed rank test. Univariate logistic regression analyses were performed to identify associations between call characteristics (bystander- and dispatcher-related predictors) and recognition of OHCA within 60 s. Results are reported as odds ratios (ORs) with 95% confidence intervals (CIs) and P-values when appropriate. A P-value <0.05 was considered statistically significant.
      A sample size calculation was based on unpublished preliminary results from another ongoing study.
      • Hardeland C.
      • Claesson A.
      • Blomberg S.N.F.
      • et al.
      Differences in out-of-hospital cardiac arrest emergency dispatch recognition and dispatcher assisted CPR instructions — a Scandinavian multicentre study.
      To test for superiority with 80% power to detect a 30% increase (from 30% to 40%) in OHCA recognized within the first minute with a significance level of 0.05, an effective sample size of 712 OHCA calls would be needed.
      All analyses were performed using IBM SPSS version 27. Ethical approval was obtained from the Swedish ethical review authority (DNR: 2019-01998).

      Results

      During 2018, SOS Alarm answered 1,140,434 emergency calls concerning health care, of which 6135 (0.5%) were later reported as OHCA incidents in the SRCR. A random sample of 1000 OHCA calls registered in the SRCR were included in the manual audit. Patient alive during call (n = 35), missing audio files (n = 30), caller unable to access and examine the victim (n = 49) and cases consisting of multiple audio files (n = 35) were excluded, leaving 851 OHCA calls eligible for the primary analysis (Fig. 1). The characteristics of the OHCA calls are shown in Table 1. A prespecified subgroup of calls where CPR was in progress (n = 95) were excluded, resulted in 756 OHCA calls eligible for a subgroup analysis (Fig. 1).
      Table 1Characteristics of out-of-hospital cardiac arrest (OHCA) calls.
      All (n = 851)OHCA recognized by
      Dispatcher (n = 715)ML (n = 729)ML, not dispatcher (n = 52)Dispatcher, not ML (n = 38)
      Patient age (years), median [IQR]72 [61, 81]72 [61, 80]72 [61, 80]73 [64, 80]70 [50, 81]
      Patient female291 (34)239 (33)247 (34)22 (42)14 (37)
      Missing6 (1)5 (1)5 (1)
      Callers female512 (60)438 (61)440 (60)26 (50)24 (63)
      Caller alone325 (38)271 (38)293 (40)25 (48)3 (8)
      Missing10 (1)1 (<0.5)1 (<0.5)2 (4)
      Caller health care professional249 (29)209 (29)204 (28)12 (23)17 (45)
      Missing2 (<0,5)2 (<0.5)2 (5)
      Caller relation to patient
       Know the victim736 (86)622 (87)642 (88)49 (94)29 (76)
       Unknown victim111 (13)90 (13)85 (12)3 (6)8 (21)
       Missing4 (0,5)3 (0,5)2 (<0.5)1 (3)
      Collapse at home622 (73)526 (74)544 (75)41 (79)23 (60)
      Missing4 (0,5)3 (0,5)3 (0,5)
      Incident witnessed
       Yes420 (49)334 (47)347 (48)34 (65)21 (55)
       No378 (44)350 (49)349 (48)11 (21)12 (32)
      Unknown/missing53 (6)31 (4)33 (4)7 (13)5 (13)
      CPR in progress95 (11)95 (13)85 (12)10 (26)
      Patient consciousness addressed
       Yes773 (91)650 (91)664 (91)48 (92)34 (90)
       No74 (9)62 (9)62 (8)4 (8)4 (10)
       Missing4 (0.5)3 (0,5)3 (0.5)
      Patient breathing addressed
       Yes830 (98)703 (98)716 (98)50 (96)37 (97)
       No19 (2)11 (2)12 (2)2 (4)1 (3)
       Missing2 (<0.5)3 (0.5)1 (<0.05)
      Cardiac aetiology480 (60)409 (61)418 (61)30 (60)21 (58)
      Call continued until arrival of ambulance
       Yes467 (55)447 (62)446 (61)10 (19)11 (29)
       No332 (39)240 (34)245 (34)31 (60)26 (68)
       Missing52 (6)28 (4)38 (5)11 (21)1 (3)
      Values are number (%) except where indicated otherwise.
      CPR, cardiopulmonary resuscitation; IQR, interquartile range; ML, machine learning framework.

      OHCA recognition

      For the primary outcome of all 851 OHCA calls, 25% (n = 213) were recognized as OHCA within 60 s by dispatchers and 36% (n = 305) by the ML. For the secondary outcomes, dispatchers recognized 40% (n = 344) cases of OHCA within 90 s versus 51% (n = 431) for ML. Median time to recognition was 94 s (IQR, 51–174 s) by dispatchers versus 72 s (IQR, 40–132 s) for the ML. OHCA was recognized at any time during the call in 84% (n = 715) by dispatchers and in 86% (n = 729) by the ML (Table 2). The ML could recognize an additional 6% (n = 52) OHCA not recognized by dispatchers, and 4% (n = 38) OHCA were recognized by dispatchers discriminated by the ML. In matched paired observations, where both the dispatcher and the ML recognized the OHCA (n = 677), the median time to recognition was 93 s (IQR, 52–171 s) by dispatchers versus 71 s (IQR, 39–128 s) for the ML. The mean difference was 28 s (SD, 92 s) (P < 0.001) (Table 2).
      Table 2Time to recognition of OHCAs in emergency calls by dispatcher and machine learning framework (ML).
      All calls (n = 851), OHCA recognized by
      DispatcherMachine learningML, not dispatcherDispatcher, not ML
      Sensitivity, n (%)715 (84)729 (86)52 (6)38 (4)
      Time to recognition (seconds), median [IQR]94 [51, 174]72 [40, 132]100 [63, 170]126 [38, 265]
      Recognition <60 s, n (%)213 (25)305 (36)12 (1)10 (1)
      Recognition <90 s, n (%)344 (40)431 (51)22 (3)12 (1)
      Paired observations, diff mean [SD] (n = 677)28 [92], P < 0.001
      Subgroup (n = 756), CPR in progress excluded, OHCA recognized by
      Sensitivity, n (%)620 (82)644 (85)52 (7)28 (4)
      Time to recognition (seconds), median [IQR]112 [62, 177]78 [43, 139]100 [63, 170]182 [80, 316]
      Recognition <60 s, n (%)145 (19)249 (33)12 (2)3 (0.4)
      Recognition <90 s, n (%)259 (34)364 (48)22 (3)5 (0.6)
      Paired observations, diff mean [SD] (n = 592)34 [93], P < 0.001
      Results are presented as n (%) or mean (SD). Mean differences between groups were analysed with Wilcoxon signed rank test. Paired observations diff mean denotes differences in time to recognition between the machine learning framework and the dispatcher for paired observations where both the dispatcher and the machine learning framework recognized OHCA.
      CPR, cardiopulmonary resuscitation; IQR, interquartile range; ML, machine learning framework; OHCA, out-of-hospital cardiac arrest; SD, standard deviation.

      Dispatcher versus ML, calls excluding ongoing CPR

      In a subgroup analysis (n = 756), when cases of CPR initiated before the emergency call were excluded (Fig. 1), the proportion of OHCA recognized within 60 s by dispatchers and the ML was 19% (n = 145) versus 33% (n = 249) respectively, and within 90 s 34% (n = 259) versus 48% (n = 364), respectively. OHCA was recognized by dispatchers in 82% (n = 620) of the calls, and by the ML in 85% (n = 644). Median time to recognition was 112 s (IQR, 62–177 s) versus 78 s (IQR, 43–139 s), respectively (Table 2). In matched paired observations (n = 592), the median time to recognition was 110 s (IQR, 61–188 s) by dispatchers versus 77 s (IQR, 42–136 s) for the ML. The mean difference was 34 s (SD, 93 s) (P < 0.001) (Table 2).

      ML recognition with different FPR settings

      Results on how different FPR settings of the ML affect the recognition rate within different time frames are shown in Fig. 2 and Table 3. Overall, ML with higher FPR settings reduced the ML time to OHCA recognition.
      Fig. 2
      Fig. 2Proportion of OHCA recognized within 30, 60 and 90 s with different false positive rate settings in the machine learning framework (all OHCA calls, n = 851). The dotted lines illustrate the proportion of OHCA recognized by the dispatcher within 30, 60 and 90 s.
      Table 3Time to recognition of OHCA by machine learning framework (ML) with different false positive rate (FPR) settings 1.0–3.0.
      All calls (n = 851)
      ML FPR 1.0ML FPR 1.5ML FPR 2.0ML FPR 2.5ML FPR 3.0
      Sensitivity, n (%)690 (81)729 (86)744 (87)754 (89)769 (90)
      Time to recognition (seconds), median [IQR]85 [49, 148]72 [40, 132]62 [35, 117]56 [32,19]52 [29, 100]
      Recognition <60 s, n (%)233 (27)305 (36)357 (42)402 (47)429 (50)
      Recognition <90 s, n (%)362 (42)431 (51)486 (57)513 (60)547 (64)
      Subgroup (756), CPR in progress excluded
      Sensitivity, n (%)610 (81)644 (85)657 (87)666 (88)681 (90)
      Time to recognition (seconds), median [IQR]91 [53, 158]78 [43, 139]67 [36, 126]60 [33, 118]55 [30, 109]
      Recognition <60 s, n (%)190 (25)249 (33)293 (39)335 (44)360 (48)
      Recognition <90 s, n (%)303 (40)364 (48)412 (54)436 (58)467 (62)
      CPR, cardiopulmonary resuscitation; IQR, interquartile range; OHCA, out-of-hospital cardiac arrest.

      Call characteristics and association with recognition rate and time to recognition

      Results using univariate logistic regression showed associations between call characteristics and recognition of OHCA within 60 s by dispatcher and the ML (Table 4). When CPR was in progress, the dispatcher was 10 times more likely to recognize the OHCA than when CPR was not initiated before the emergency call (95% CI, 6.55–17.17). The corresponding results for the ML was 2 times more likely (95% CI, 1.89–4.52). ML recognition was positively associated in calls were the bystander knew the patient versus when the patient was unfamiliar (OR, 1.97; 95% CI, 1.25–3.12). When the OHCA was witnessed by a bystander, both the dispatchers and the ML were less likely to recognize the OHCA (OR, 0.42; 95% CI, 0.30–0.58) compared with incidents that were unwitnessed (OR, 0.43; 95% CI, 0.32–0.58). Recognition by the ML was negatively associated when the caller/bystander was a health care professional versus when the caller was a layperson (OR, 0.66; 95% CI, 0.48–0.91).
      Table 4Associations between call characteristics and recognition of OHCA within 60 s, analysed by univariate logistic regression analyses.
      n/totalRecognition by dispatcherRecognition by machine learning
      Odds ratio95% CIP valueOdds ratio95% CIP value
      Bystanders
      CPR in progress95/85110.616.55–17.17<0.0012.921.89–4.52<0.001
      Female512/8511.070.78–1.480.6451.280.95–1.710.093
      Known the patient736/8511.160.73–1.860.5201.971.25–3.120.003
      Health care professional249/8511.210.87–1.700.2460.660.48–0.910.011
      Alone on site325/8480.790.57–1.090.1601.320.99–1.760.054
      Witnessed OHCA420/8390.420.30–0.58<0.0010.430.32–0.58<0.001
      Dispatchers
      Consciousness addressed773/8510.890.53–1.510.680.830.51–1.340.451
      Breathing addressed830/8513.230.74–14.010.1161.810.65–4.990.251
      An odds ratio greater than one means the factor was positively associated with recognition, and odds ratios less than one mean the factor was associated with failing to recognize the OHCA.
      CI, confidence interval; CPR, cardiopulmonary resuscitation; OHCA, out-of-hospital cardiac arrest.
      When breathing was addressed during the call, the dispatcher was four times more likely to recognize the OHCA versus when breathing not addressed (OR, 4.15; 95% CI, 1.71–10.05). Corresponding results for the ML were similar (OR, 3.86; 95% CI, 1.56–9.53) (Supplementary Table 1).

      Priority 1–4 calls (cohort B)

      In the random sample of 85,205 calls, labelled as no OHCA, the ML suspected an OHCA at any time in 0.8% of the calls (n = 697). Of the ML-suspected OHCAs, a manual audit of 100 randomly selected calls showed one OHCA (also recognized by the dispatcher); in 9% (n = 9) of calls, the dispatchers also suspected an OHCA; 74% (n = 74) of calls involved an unconscious patient; and in 4% (n = 4) of calls, the dispatchers should have suspected an OHCA based on the information presented by the caller.

      Discussion

      The main findings are threefold. First, the ML recognized a higher proportion of OHCAs within the first minute compared with dispatchers. Second, when emergency calls with ongoing CPR were excluded, ML proved to be even more beneficial. Third, small adjustments in the FPR settings of the ML affect the time to recognition.

      OHCA recognition

      Even though our study used an ML with a lower FPR rate compared with Blomberg et al., the results of the present study confirm previous findings that ML, based on historical audio files, is faster and has a higher sensitivity than a dispatcher.
      • Blomberg S.N.
      • Folke F.
      • Ersbøll A.K.
      • et al.
      Machine learning as a supportive tool to recognize cardiac arrest in emergency calls.
      The ML has the potential to be a supportive tool to aid dispatchers in recognizing OHCAs during live emergency calls and thus, potentially increasing survival rates.
      • Blomberg S.N.
      • Folke F.
      • Ersbøll A.K.
      • et al.
      Machine learning as a supportive tool to recognize cardiac arrest in emergency calls.

      FPR setting

      ML frameworks with a low FPR setting result in fewer recognized OHCAs. When the FPR setting is increased, the performance of the ML improves. In a real-life setting, a high FPR may lead to a false ML interpretation, resulting in emergency calls that do not involve an OHCA. This may result in an increased unwanted dispatch level, depleting available resources for other patients in need of EMS. There is also a risk dispatchers lose trust in the ML alerts and start ignoring its alerts in future emergency calls. One solution could be that the ML provides two types of alerts, with different FPR settings: (1) a pre-alert, set on a higher FPR setting (e.g., FPR 2.0–2.5), with advice to ensure consciousness and breathing, and (2) suspected OHCA alert, set on a lower FPR setting (e.g., FPR 1.5).

      Call characteristics and recognition

      Witnessed OHCA has been found to be associated with less chance of or prolonged time to recognition compared with unwitnessed cases due to the presence of agonal breathing and/or seizures.
      • Drennan I.R.
      • Geri G.
      • Brooks S.
      • Couper K.
      • et al.
      Diagnosis of out-of-hospital cardiac arrest by emergency medical dispatch: a diagnostic systematic review.
      • Schwarzkoph M.
      • Yin L.
      • Hergert L.
      • Drucker C.
      • Counts C.R.
      • Eisenberg M.
      Seizure-like presentation in OHCA creates barriers to dispatch recognition of cardiac arrest.
      • Bång A.
      • Herlitz J.
      • Martinell S.
      Interaction between emergency medical dispatcher and caller in suspected out-of-hospital cardiac arrest calls with focus on agonal breathing. A review of 100 tape recordings of true cardiac arrest cases.
      • Bohm K.
      • Stålhandske B.
      • Rosenqvist M.
      • Ulfvarson J.
      • Hollenberg J.
      • Svensson L.
      Tuition of emergency medical dispatchers in the recognition of agonal respiration increases the use of telephone assisted CPR.
      • Roppolo L.P.
      • Westfall A.
      • Pepe P.E.
      • et al.
      Dispatcher assessments for agonal breathing improve detection of cardiac arrest.
      When the OHCA was witnessed by a bystander, both the dispatchers and the ML were less likely to recognize the OHCA compared with incidents that were unwitnessed.

      Priority 1–4 calls

      To evaluate its specificity, the ML was also exposed to a random sample of emergency calls labelled as non-OHCA calls (cohort B). In cohort B, the ML suspected an OHCA in 0.8% (n = 697) of 85,205 emergency calls. The manual audit of the ML-suspected OHCAs showed that the most common condition (74%) in these patients was unconsciousness. In a real live setting, a supplementary question from the dispatcher, regarding whether the victim is breathing normally would result in the dispatcher being able to reject the suspicion.
      Only 1% of medical emergency calls to EMDC involve an OHCA.
      • Viereck S.
      • Møller T.P.
      • Ersbøll A.K.
      • et al.
      Recognising out-of-hospital cardiac arrest during emergency calls increases bystander cardiopulmonary resuscitation and survival.
      Therefore, dispatchers rarely handle this type of emergency call. This might have a negative effect on dispatchers’ inclination to suspect an OHCA and their ability to recognize it. Hardeland et al. showed how education, targeted simulations and feedback on OHCA recognition improves dispatchers’ performance in recognizing OHCAs (89%–95%) and reduces delayed OHCA recognition (21% to 6%).
      • Hardeland C.
      • Skåre C.
      • Kramer-Johansen J.
      • et al.
      Targeted simulation and education to improve cardiac arrest recognition and telephone assisted CPR in an emergency medical communication centre.
      Education and training need to be implemented continuously in the organization to have an effect, which is time and resource consuming. A modified education and training approach in combination with the ML as a supportive tool could improve dispatchers’ performance in a similar way.

      Future studies

      We suggest that the optimal FPR setting of the ML needs to be evaluated in a prospective study. Dispatchers’ experience of integrating with the ML should also be examined so ML is used in an optimal way in a real-life setting.

      Limitations

      Dispatchers could have recognized an OHCA at an earlier time point, but not verbally communicated the need for CPR due to circumstances present at the scene.
      • Olasveengen T.M.
      • Hardeland C.
      Man vs. machine? The future of emergency medical dispatching.
      It is unclear what clinical significance a time difference of 28–34 s, as shown in this study, has on patients survival, but it can be assumed that it may play a role in bystander CPR situations.
      • Strömsöe A.
      • Afzelius S.
      • Axelsson C.
      • et al.
      Improvements in logistics could increase survival after out-of-hospital cardiac arrest in Sweden.
      The ML was tested on two cohorts thus we cannot calculate negative or positive predictive values (PPV). However, with the low proportion of OHCA calls overall, even with high sensitivity and specificity, the PPV would probably be low. As mentioned earlier, a possible solution for this is a two-step process with pre-alert and OHCA alert. We are not able to report dispatchers’ specificity, because it would require considerable further manual audit of calls. At some EMDCs, the call is connected to a public safety answering point, in Sweden, all calls are handled by the dispatcher directly. Thus, it is difficult to compare the time for recognition between studies. Dispatchers trained in a different way or different dispatch systems could affect the results in different directions.
      • Hardeland C.
      • Skåre C.
      • Kramer-Johansen J.
      • et al.
      Targeted simulation and education to improve cardiac arrest recognition and telephone assisted CPR in an emergency medical communication centre.
      • Hardeland C.
      • Olasveengen T.M.
      • Lawrence R.
      • et al.
      Comparison of Medical Priority Dispatch (MPD) and Criteria Based Dispatch (CBD) relating to cardiac arrest calls.

      Conclusion

      The ML framework recognized a higher proportion of OHCAs within the first minute compared with dispatchers and has the potential to be a supportive tool during live emergency calls. Optimal FPR settings need to be evaluated in a prospective study.

      Funding

      Corti Company funded the programming and language training phase of machine learning. Funding was received from Laerdal Foundation and Riksförbundet HjärtLung. None had any role in the design, analysis or writing of the article.

      Conflict of interest

      None.

      CRediT authorship contribution statement

      Fredrik Byrsell: Conceptualization, Methodology, Data curation, Formal analysis, Writing - original draft, Writing - review & editing. Andreas Claesson: Conceptualization, Methodology, Writing - review & editing. Mattias Ringh: Conceptualization, Methodology, Writing - review & editing. Leif Svensson: Conceptualization, Methodology, Writing - review & editing. Martin Jonsson: Formal analysis, Writing - review & editing. Per Nordberg: Writing - review & editing. Sune Forsberg: Writing - review & editing. Jacob Hollenberg: Conceptualization, Writing - review & editing. Anette Nord: Conceptualization, Methodology, Formal analysis, Writing - review & editing.

      Acknowledgements

      We would like to thank SOS Alarm AB for helping us with the voice log retrievals and special thanks to Jenny Hernerud for helping us with the manual audit. We also thank Corti ApS who created the ML framework. They had no role in the analysis or writing of the article.

      Appendix A. Supplementary data

      The following are Supplementary data to this article:

      References

        • Gräsner J.T.
        • Lefering R.
        • Koster R.W.
        EuReCa ONE-27 Nations, ONE Europe, ONE Registry: a prospective one month analysis of out-of-hospital cardiac arrest outcomes in 27 countries in Europe.
        Resuscitation. 2016; 105: 188-195https://doi.org/10.1016/j.resuscitation.2016.06.004
        • Gräsner J.T.
        • Wnent J.
        • Herlitz J.
        • et al.
        Survival after out-of-hospital cardiac arrest in Europe - results of the EuReCa TWO study.
        Resuscitation. 2020; 148: 218-226https://doi.org/10.1016/j.resuscitation.2019.12.042
        • Herlitz J.
        Hjärt- och lungräddningsregistrets årsrapport.
        2019 (Accessed 26 November 2020, at https://shlr.registercentrum.se/)
        • Perkins G.D.
        • Handley A.J.
        • Koster R.W.
        • et al.
        European Resuscitation Council Guidelines for Resuscitation 2015. Section 2. Adult basic life support and automated external defibrillation.
        Resuscitation. 2015; 95: 81-99https://doi.org/10.1016/j.resuscitation.2015.07.015
        • Hollenberg J.
        • Riva G.
        • Bohm K.
        • et al.
        Dual dispatch early defibrillation in out-of-hospital cardiac arrest: the SALSA-pilot.
        Eur Heart J. 2009; 30: 1781-1789https://doi.org/10.1093/eurheartj/ehp177
        • Hasselqvist-Ax I.
        • Nordberg P.
        • Herlitz J.
        • et al.
        Dispatch of firefighters and police officers in out-of-hospital cardiac arrest: a nationwide prospective cohort trial using propensity score analysis.
        J Am Heart Assoc. 2017; 6e005873https://doi.org/10.1161/JAHA.117.005873
        • Ringh M.
        • Jonsson M.
        • Nordberg P.
        • et al.
        Survival after public access defibrillation in Stockholm, Sweden — a striking success.
        Resuscitation. 2015; 91: 1-7https://doi.org/10.1016/j.resuscitation.2015.02.032
        • Drennan I.R.
        • Geri G.
        • Brooks S.
        • Couper K.
        • et al.
        Diagnosis of out-of-hospital cardiac arrest by emergency medical dispatch: a diagnostic systematic review.
        Resuscitation. 2021; 159: 85-96https://doi.org/10.1016/j.resuscitation.2020.11.025
        • Bürger A.
        • Wnent J.
        • Bohn A.
        • et al.
        The effect of ambulance response time on survival following out-of-hospital cardiac arrest.
        Dtsch Arztebl Int. 2018; 115: 541-548https://doi.org/10.3238/arztebl.2018.0541
        • Hara M.
        • Hayashi K.
        • Hikoso S.
        • Sakata Y.
        • Kitamura T.
        Different impacts of time from collapse to first cardiopulmonary resuscitation on outcomes after witnessed out-of-hospital cardiac arrest in adults, circulation: cardiovascular quality and outcomes.
        Circ Cardiovasc Qual Outcomes. 2015; 8: 277-284https://doi.org/10.1161/CIRCOUTCOMES.115.001864
        • Holmén J.
        • Herlitz J.
        • Ricksten S.E.
        • et al.
        Shortening ambulance response time increases survival in out-of-hospital cardiac arrest.
        J Am Heart Assoc. 2020; 9e017048https://doi.org/10.1161/JAHA.120.017048
        • Rajan S.
        • Wissenberg M.
        • Folke F.
        • et al.
        Association of bystander cardiopulmonary resuscitation and survival according to ambulance response times after out-of-hospital cardiac arrest.
        Circulation. 2016; 134: 2095-2104https://doi.org/10.1161/CIRCULATIONAHA.116.024400
        • Viereck S.
        • Møller T.P.
        • Ersbøll A.K.
        • et al.
        Recognising out-of-hospital cardiac arrest during emergency calls increases bystander cardiopulmonary resuscitation and survival.
        Resuscitation. 2017; 115: 141-147https://doi.org/10.1016/j.resuscitation.2017.04.006
        • Viereck S.
        • Møller T.P.
        • Rothman J.P.
        • Folke F.
        • Lippert F.K.
        Recognition of out-of-hospital cardiac arrest during emergency calls — a systematic review of observational studies.
        Scand J Trauma Resusc Emerg Med. 2017; 25: 9https://doi.org/10.1186/s13049-017-0350-8
        • Møller T.P.
        • Andréll C.
        • Viereck S.
        • Todorova L.
        • Friberg H.
        • Lippert F.K.
        Recognition of out-of-hospital cardiac arrest by medical dispatchers in emergency medical dispatch centres in two countries.
        Resuscitation. 2016; 109: 1-8https://doi.org/10.1016/j.resuscitation.2016.09.012
        • Schwarzkoph M.
        • Yin L.
        • Hergert L.
        • Drucker C.
        • Counts C.R.
        • Eisenberg M.
        Seizure-like presentation in OHCA creates barriers to dispatch recognition of cardiac arrest.
        Resuscitation. 2020; 156: 230-236https://doi.org/10.1016/j.resuscitation.2020.06.036
        • Breckwoldt J.
        • Schloesser S.
        • Arntz H.-R.
        Perceptions of collapse and assessment of cardiac arrest by bystanders of out-of-hospital cardiac arrest (OOHCA).
        Resuscitation. 2009; 80: 1108-1113https://doi.org/10.1016/j.resuscitation.2009.06.028
        • Hardeland C.
        • Claesson A.
        • Blomberg S.N.F.
        • et al.
        Differences in out-of-hospital cardiac arrest emergency dispatch recognition and dispatcher assisted CPR instructions — a Scandinavian multicentre study.
        Resuscitation. 2019; 142: e13https://doi.org/10.1016/j.resuscitation.2019.06.041
        • Kurz M.C.
        • Bobrow B.J.
        • Buckingham J.
        • et al.
        Telecommunicator cardiopulmonary resuscitation: a policy statement from the American Heart Association.
        Circulation. 2020; : E686-700https://doi.org/10.1161/CIR.0000000000000744
        • De Fauw J.
        • Ledsam J.R.
        • Romera-Paredes B.
        • et al.
        Clinically applicable deep learning for diagnosis and referral in retinal disease.
        Nat Med. 2018; 24: 1342-1350https://doi.org/10.1038/s41591-018-0107-6
        • Gulshan V.
        • Peng L.
        • Coram M.
        • et al.
        Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.
        JAMA. 2016; 316: 2402-2410https://doi.org/10.1001/jama.2016.17216
        • Ceccarelli F.
        • Sciandrone M.
        • Perricone C.
        • et al.
        Prediction of chronic damage in systemic lupus erythematosus by using machine-learning models.
        PLoS One. 2017; 12e0174200https://doi.org/10.1371/journal.pone.0174200
        • Esteva A.
        • Kuprel B.
        • Novoa R.A.
        • et al.
        Dermatologist-level classification of skin cancer with deep neural networks.
        Nat Publ Gr. 2017; 542: 115-118https://doi.org/10.1038/nature21056
        • Blomberg S.N.
        • Folke F.
        • Ersbøll A.K.
        • et al.
        Machine learning as a supportive tool to recognize cardiac arrest in emergency calls.
        Resuscitation. 2019; 138: 322-329https://doi.org/10.1016/j.resuscitation.2019.01.015
        • Kwon J.
        • Lee Y.
        • Lee Y.
        • Lee S.
        • Park J.
        An algorithm based on deep learning for predicting in‐hospital cardiac arrest.
        J Am Heart Assoc. 2018; 7https://doi.org/10.1161/JAHA.118.008678
        • Stewart J.
        • Sprivulis P.
        • Dwivedi G.
        Artificial intelligence and machine learning in emergency medicine.
        Emerg Med Australas. 2018; 30: 870-874https://doi.org/10.1111/1742-6723.13145
      1. AI for Patient Consultations.
        2020 (Accessed 8 December 2020, at https://www.corti.ai/)
        • Borgholt L.
        • Havtorn J.D.
        • Agić Ž
        • Søgaard A.
        • Maaløe L.
        • Igel C.
        Do end-to-end speech recognition models care about context?.
        in: INTERSPEECH, October 25–29, 2020, Shanghai, China2020: 4352-4356https://doi.org/10.21437/interspeech.2020-1750
      2. Den Norske lægeforening. Norsk indeks for medisinsk nødhjelp. n.d. (Accessed 26 November 2020, at https://www.helsedirektoratet.no/veiledere/norsk-indeks-for-medisinsk-nodhjelp/Norsk%20indeks%20for%20medisinsk%20n%C3%B8dhjelp.pdf/_/attachment/inline/c9aa280e-3e84-4197-8743-36f5dd934b8b:f649297bce76e95aed934803c67384569b027d61/Norsk%20indeks%20for%20medisinsk%20n%C3%B8dhjelp.pdf).

        • Jacobs I.
        • Nadkarni V.
        • Bahr J.
        • et al.
        Cardiac arrest and cardiopulmonary resuscitation outcome reports: update and simplification of the Utstein templates for resuscitation registries. A statement for healthcare professionals from a task force of the International Liaison Committee on Resuscitation (American Heart Association, European Resuscitation Council, Australian Resuscitation Council, New Zealand Resuscitation Council.
        Circulation. 2004; 110: 3385-3397https://doi.org/10.1161/01.CIR.0000147236.85306.15
        • Strömsöe A.
        • Svensson L.
        • Axelsson ÅB.
        • Göransson K.
        • Todorova L.
        • Herlitz J.
        Validity of reported data in the Swedish Cardiac Arrest Register in selected parts in Sweden.
        Resuscitation. 2013; 84: 952-956https://doi.org/10.1016/j.resuscitation.2012.12.026
        • Emilsson L.
        • Lindahl B.
        • Köster M.
        • Lambe M.
        • Ludvigsson J.F.
        Review of 103 Swedish healthcare quality registries.
        J Intern Med. 2015; 277: 94-136https://doi.org/10.1111/joim.12303
      3. CARES Dispather-Assisted CPR Data Dictionary. n.d. (Accessed 26 November 2020, at https://mycares.net/sitepages/uploads/2015/CARES%20Dispatcher-Assisted%20CPR%20Data%20Dictionary%2012-6-15.pdf).

        • Bång A.
        • Herlitz J.
        • Martinell S.
        Interaction between emergency medical dispatcher and caller in suspected out-of-hospital cardiac arrest calls with focus on agonal breathing. A review of 100 tape recordings of true cardiac arrest cases.
        Resuscitation. 2003; 56: 25-34https://doi.org/10.1016/S0300-9572(02)00278-2
        • Bohm K.
        • Stålhandske B.
        • Rosenqvist M.
        • Ulfvarson J.
        • Hollenberg J.
        • Svensson L.
        Tuition of emergency medical dispatchers in the recognition of agonal respiration increases the use of telephone assisted CPR.
        Resuscitation. 2009; 80: 1025-1028https://doi.org/10.1016/j.resuscitation.2009.06.004
        • Roppolo L.P.
        • Westfall A.
        • Pepe P.E.
        • et al.
        Dispatcher assessments for agonal breathing improve detection of cardiac arrest.
        Resuscitation. 2009; 80: 769-772https://doi.org/10.1016/j.resuscitation.2009.04.013
        • Hardeland C.
        • Skåre C.
        • Kramer-Johansen J.
        • et al.
        Targeted simulation and education to improve cardiac arrest recognition and telephone assisted CPR in an emergency medical communication centre.
        Resuscitation. 2017; 114: 21-26https://doi.org/10.1016/j.resuscitation.2017.02.013
        • Olasveengen T.M.
        • Hardeland C.
        Man vs. machine? The future of emergency medical dispatching.
        Resuscitation. 2019; 138: 304-305https://doi.org/10.1016/j.resuscitation.2019.03.005
        • Strömsöe A.
        • Afzelius S.
        • Axelsson C.
        • et al.
        Improvements in logistics could increase survival after out-of-hospital cardiac arrest in Sweden.
        J Intern Med. 2013; 273: 622-627https://doi.org/10.1111/joim.12041
        • Hardeland C.
        • Olasveengen T.M.
        • Lawrence R.
        • et al.
        Comparison of Medical Priority Dispatch (MPD) and Criteria Based Dispatch (CBD) relating to cardiac arrest calls.
        Resuscitation. 2014; 85: 612-616https://doi.org/10.1016/j.resuscitation.2014.01.029