If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Fast recognition of out-of-hospital cardiac arrest (OHCA) by dispatchers might increase survival. The aim of this observational study of emergency calls was to (1) examine whether a machine learning framework (ML) can increase the proportion of calls recognizing OHCA within the first minute compared with dispatchers, (2) present the performance of ML with different false positive rate (FPR) settings, (3) examine call characteristics influencing OHCA recognition.
Methods
ML can be configured with different FPR settings, i.e., more or less inclined to suspect an OHCA depending on the predefined setting. ML OHCA recognition within the first minute is evaluated with a 1.5 FPR as the primary endpoint, and other FPR settings as secondary endpoints. ML was exposed to a random sample of emergency calls from 2018. Voice logs were manually audited to evaluate dispatchers time to recognition.
Results
Of 851 OHCA calls, the ML recognized 36% (n = 305) within 1 min compared with 25% (n = 213) by dispatchers. The recognition rate at any time during the call was 86% for ML and 84% for dispatchers, with a median time to recognition of 72 versus 94 s. OHCA recognized by both ML and dispatcher showed a 28 s mean difference in favour of ML (P < 0.001). ML with higher FPR settings reduced recognition times.
Conclusion
ML recognized a higher proportion of OHCA within the first minute compared with dispatchers and has the potential to be a supportive tool during emergency calls. The optimal FPR settings need to be evaluated in a prospective study.
EuReCa ONE-27 Nations, ONE Europe, ONE Registry: a prospective one month analysis of out-of-hospital cardiac arrest outcomes in 27 countries in Europe.
In Sweden, the incidence of OHCA is 60 per 100,000 population, and the incidence of resuscitation by bystanders and first responders is 45 per 100,000 population.
Cardiopulmonary resuscitation (CPR) and use of an automated external defibrillator (AED) by lay persons and first responders can increase survival in 7 of 10 cases.
Recognition of OHCA by dispatchers is time critical; the earlier recognition, the earlier start of telephone assisted CPR (T-CPR) can be initiated in parallel with dispatch of emergency medical services (EMS).
Different impacts of time from collapse to first cardiopulmonary resuscitation on outcomes after witnessed out-of-hospital cardiac arrest in adults, circulation: cardiovascular quality and outcomes.
Furthermore, witnessed OHCA and patients presenting with agonal breathing and/or seizures have been shown to introduce complexity to the recognition process.
Delay in recognition and the proportion of correct recognition vary greatly. In a review of 16 observational studies, the median sensitivity for recognition of OHCA was 74% (range, 14%–97%).
A multicentre study including Emergency Medical Dispatch Centres (EMDC) from Copenhagen, Oslo and Stockholm showed 77% OHCA recognition in Copenhagen, 79% in Stockholm and 96% in Oslo.
There is an urgent need for new and novel methods that can shorten the time for OHCA recognition and start of treatment. Machine learning (ML) with artificial intelligence can be used to assist with clinical decisions.
A recent study demonstrated that ML has the potential to recognize OHCA during emergency calls with higher sensitivity and specificity compared with dispatchers.
The performance of the ML in recognizing OHCAs in emergency calls is dependent on its predefined false positive rate (FPR) setting.
The primary aim of this observational study of Swedish emergency calls is to examine whether an ML framework can increase the proportion of recognized OHCA calls within the first minute compared with dispatchers. The secondary aims are to present ML performance with different FPR settings and to examine the call characteristics influencing OHCA recognition.
Methods
In this retrospective, registry-based study, ML was used to assist in recognizing OHCA in voice logs of Swedish emergency calls from 2018.
Machine learning framework
Machine learning is a way of programming computer software. Instead of programming instructions about what action to be executed, the computer utilizes complex statistical models to learn from large amounts of data to then understand problems and solve them.
The framework contains two ML models: an automatic speech recognition (ASR) model transcribes speech to text and an OHCA detection model predicts OHCA events from transcribed speech in real time.
Automatic speech recognition
ASR is a deep neural network using a model based on Connectionist Temporal Classification.
To train the ASR model in the Swedish language, a total of 45 h of uniform random selected Swedish emergency calls from 2015 concerning all types of emergencies were manually transcribed into written text. All text files were then used as labels for the ASR model to be trained in understanding the Swedish language. After the training phase, a validation and tuning phase was conducted.
OHCA detection model
The OHCA detection model is a densely connected deep neural network model to detect an OHCA in the transcribed text produced by the ASR model. For each second of raw audio, the ML predicts whether there is an OHCA from the accumulated audio sequence. In training the OHCA detection model, we used 3944 calls labelled as OHCA reported to the Swedish Registry of Cardiopulmonary Resuscitation (SRCR) by the EMS during 2016 and 39,888 calls labelled as no OHCA.
The ML can be configured with different FPR settings, meaning the ML can be more or less inclined to suspect an OHCA depending on its predefined setting. This study has a predefined 1.5% FPR setting in the primary outcome analysis. Different FPR settings and their impact on the outcome were also analysed.
Dispatchers
When dialling the Swedish national emergency number, the call is received by a dispatcher at one of the 15 EMDCs run by SOS Alarm AB. In Sweden, the dispatchers at EMDCs handle emergency calls concerning a medical emergency from start to finish, except in three regions where the call is redirected to local EMDCs run by local EMS organizations. Dispatchers at SOS Alarm undergo a general training program for 14 weeks before entering an answering position taking emergency calls. Annually, all dispatchers must be certified to continue answering emergency calls. Dispatchers and registered nurses at SOS Alarm are supported in their medical decision making by a criteria-based medical index.
EMS in Sweden report OHCAs to the SRCR when bystanders or EMS have initiated resuscitation efforts. The SRCR adheres to Utstein template of OHCA registration.
Cardiac arrest and cardiopulmonary resuscitation outcome reports: update and simplification of the Utstein templates for resuscitation registries. A statement for healthcare professionals from a task force of the International Liaison Committee on Resuscitation (American Heart Association, European Resuscitation Council, Australian Resuscitation Council, New Zealand Resuscitation Council.
The ML was tested on two uniform random cohorts to evaluate its sensitivity and specificity: (a) validated OHCA calls, and (b) priority 1–4 calls (no OHCA).
OHCA calls: cohort A
In 2018, a total of 6135 cases of OHCA were registered by EMS personnel to the SRCR. After excluding OHCA calls witnessed by the EMS personnel, calls not available at the time of extraction and calls handled by care providers from regions where SOS Alarm does not assess the medical urgency and apply a dispatch code, a random sample were extracted for inclusion. In addition, in a subgroup analysis, we excluded calls where CPR was in progress (Fig. 1).
Fig. 1Flowchart of the study: OHCA calls (cohort A).
In 2018, 1,140,434 emergency calls concerning health care were answered by EMDCs in Sweden. Cohort B consisted of a random sample (n = 85,205) of priority 1–4 calls, with the following calls excluded: validated OHCA registered in the SRCR, calls logged with a dispatch code or manual note that could possibly involve an OHCA, lifesavers activated through mobile text message, calls from regions with local EMDCs, and calls that had more than one case folder or with more than one emergency call to minimize the risk of duplicates in the sample (Supplementary Fig. 1).
Manual audit
To evaluate dispatchers’ performance of OHCA, a manual audit of all voice logs in cohort A was performed by two investigators. A modified version of the Cardiac Arrest Registry to Enhance Survival (CARES) dispatcher-assisted CPR data dictionary was used to extract data from the voice logs.
Dispatchers’ recognition of an OHCA was defined as when dispatchers mentioned the presence of an OHCA or the need for CPR. Time to recognition was measured from the time the dispatcher answered the emergency call until the definition of OHCA recognition was achieved.
In addition, a random sample of OHCAs suspected by the ML (false positives) in cohort B were manually audited to evaluate if there could be unregistered OHCA calls present.
Outcome measures
The primary endpoint was the proportion (%) of OHCAs recognized within 60 s, ML versus dispatcher. Secondary endpoints were the proportion (%) of OHCAs recognized within 90 s, median time to recognition, recognition rate at any timepoint during the call, time difference between OHCA recognized by both ML and dispatcher (paired observation), test of different FPR settings and characteristics influencing OHCA recognition.
Statistical analysis
Data are presented as the proportion (%), median (interquartile range) or mean (standard deviation). For paired observations, the mean differences in time to recognition between groups were analysed with the Wilcoxon signed rank test. Univariate logistic regression analyses were performed to identify associations between call characteristics (bystander- and dispatcher-related predictors) and recognition of OHCA within 60 s. Results are reported as odds ratios (ORs) with 95% confidence intervals (CIs) and P-values when appropriate. A P-value <0.05 was considered statistically significant.
A sample size calculation was based on unpublished preliminary results from another ongoing study.
To test for superiority with 80% power to detect a 30% increase (from 30% to 40%) in OHCA recognized within the first minute with a significance level of 0.05, an effective sample size of 712 OHCA calls would be needed.
All analyses were performed using IBM SPSS version 27. Ethical approval was obtained from the Swedish ethical review authority (DNR: 2019-01998).
Results
During 2018, SOS Alarm answered 1,140,434 emergency calls concerning health care, of which 6135 (0.5%) were later reported as OHCA incidents in the SRCR. A random sample of 1000 OHCA calls registered in the SRCR were included in the manual audit. Patient alive during call (n = 35), missing audio files (n = 30), caller unable to access and examine the victim (n = 49) and cases consisting of multiple audio files (n = 35) were excluded, leaving 851 OHCA calls eligible for the primary analysis (Fig. 1). The characteristics of the OHCA calls are shown in Table 1. A prespecified subgroup of calls where CPR was in progress (n = 95) were excluded, resulted in 756 OHCA calls eligible for a subgroup analysis (Fig. 1).
Table 1Characteristics of out-of-hospital cardiac arrest (OHCA) calls.
All (n = 851)
OHCA recognized by
Dispatcher (n = 715)
ML (n = 729)
ML, not dispatcher (n = 52)
Dispatcher, not ML (n = 38)
Patient age (years), median [IQR]
72 [61, 81]
72 [61, 80]
72 [61, 80]
73 [64, 80]
70 [50, 81]
Patient female
291 (34)
239 (33)
247 (34)
22 (42)
14 (37)
Missing
6 (1)
5 (1)
5 (1)
–
–
Callers female
512 (60)
438 (61)
440 (60)
26 (50)
24 (63)
Caller alone
325 (38)
271 (38)
293 (40)
25 (48)
3 (8)
Missing
10 (1)
1 (<0.5)
1 (<0.5)
2 (4)
–
Caller health care professional
249 (29)
209 (29)
204 (28)
12 (23)
17 (45)
Missing
2 (<0,5)
2 (<0.5)
–
–
2 (5)
Caller relation to patient
Know the victim
736 (86)
622 (87)
642 (88)
49 (94)
29 (76)
Unknown victim
111 (13)
90 (13)
85 (12)
3 (6)
8 (21)
Missing
4 (0,5)
3 (0,5)
2 (<0.5)
–
1 (3)
Collapse at home
622 (73)
526 (74)
544 (75)
41 (79)
23 (60)
Missing
4 (0,5)
3 (0,5)
3 (0,5)
–
–
Incident witnessed
Yes
420 (49)
334 (47)
347 (48)
34 (65)
21 (55)
No
378 (44)
350 (49)
349 (48)
11 (21)
12 (32)
Unknown/missing
53 (6)
31 (4)
33 (4)
7 (13)
5 (13)
CPR in progress
95 (11)
95 (13)
85 (12)
–
10 (26)
Patient consciousness addressed
Yes
773 (91)
650 (91)
664 (91)
48 (92)
34 (90)
No
74 (9)
62 (9)
62 (8)
4 (8)
4 (10)
Missing
4 (0.5)
3 (0,5)
3 (0.5)
–
–
Patient breathing addressed
Yes
830 (98)
703 (98)
716 (98)
50 (96)
37 (97)
No
19 (2)
11 (2)
12 (2)
2 (4)
1 (3)
Missing
2 (<0.5)
3 (0.5)
1 (<0.05)
–
–
Cardiac aetiology
480 (60)
409 (61)
418 (61)
30 (60)
21 (58)
Call continued until arrival of ambulance
Yes
467 (55)
447 (62)
446 (61)
10 (19)
11 (29)
No
332 (39)
240 (34)
245 (34)
31 (60)
26 (68)
Missing
52 (6)
28 (4)
38 (5)
11 (21)
1 (3)
Values are number (%) except where indicated otherwise.
For the primary outcome of all 851 OHCA calls, 25% (n = 213) were recognized as OHCA within 60 s by dispatchers and 36% (n = 305) by the ML. For the secondary outcomes, dispatchers recognized 40% (n = 344) cases of OHCA within 90 s versus 51% (n = 431) for ML. Median time to recognition was 94 s (IQR, 51–174 s) by dispatchers versus 72 s (IQR, 40–132 s) for the ML. OHCA was recognized at any time during the call in 84% (n = 715) by dispatchers and in 86% (n = 729) by the ML (Table 2). The ML could recognize an additional 6% (n = 52) OHCA not recognized by dispatchers, and 4% (n = 38) OHCA were recognized by dispatchers discriminated by the ML. In matched paired observations, where both the dispatcher and the ML recognized the OHCA (n = 677), the median time to recognition was 93 s (IQR, 52–171 s) by dispatchers versus 71 s (IQR, 39–128 s) for the ML. The mean difference was 28 s (SD, 92 s) (P < 0.001) (Table 2).
Table 2Time to recognition of OHCAs in emergency calls by dispatcher and machine learning framework (ML).
All calls (n = 851), OHCA recognized by
Dispatcher
Machine learning
ML, not dispatcher
Dispatcher, not ML
Sensitivity, n (%)
715 (84)
729 (86)
52 (6)
38 (4)
Time to recognition (seconds), median [IQR]
94 [51, 174]
72 [40, 132]
100 [63, 170]
126 [38, 265]
Recognition <60 s, n (%)
213 (25)
305 (36)
12 (1)
10 (1)
Recognition <90 s, n (%)
344 (40)
431 (51)
22 (3)
12 (1)
Paired observations, diff mean [SD] (n = 677)
28 [92], P < 0.001
–
–
Subgroup (n = 756), CPR in progress excluded, OHCA recognized by
Sensitivity, n (%)
620 (82)
644 (85)
52 (7)
28 (4)
Time to recognition (seconds), median [IQR]
112 [62, 177]
78 [43, 139]
100 [63, 170]
182 [80, 316]
Recognition <60 s, n (%)
145 (19)
249 (33)
12 (2)
3 (0.4)
Recognition <90 s, n (%)
259 (34)
364 (48)
22 (3)
5 (0.6)
Paired observations, diff mean [SD] (n = 592)
34 [93], P < 0.001
–
–
Results are presented as n (%) or mean (SD). Mean differences between groups were analysed with Wilcoxon signed rank test. Paired observations diff mean denotes differences in time to recognition between the machine learning framework and the dispatcher for paired observations where both the dispatcher and the machine learning framework recognized OHCA.
In a subgroup analysis (n = 756), when cases of CPR initiated before the emergency call were excluded (Fig. 1), the proportion of OHCA recognized within 60 s by dispatchers and the ML was 19% (n = 145) versus 33% (n = 249) respectively, and within 90 s 34% (n = 259) versus 48% (n = 364), respectively. OHCA was recognized by dispatchers in 82% (n = 620) of the calls, and by the ML in 85% (n = 644). Median time to recognition was 112 s (IQR, 62–177 s) versus 78 s (IQR, 43–139 s), respectively (Table 2). In matched paired observations (n = 592), the median time to recognition was 110 s (IQR, 61–188 s) by dispatchers versus 77 s (IQR, 42–136 s) for the ML. The mean difference was 34 s (SD, 93 s) (P < 0.001) (Table 2).
ML recognition with different FPR settings
Results on how different FPR settings of the ML affect the recognition rate within different time frames are shown in Fig. 2 and Table 3. Overall, ML with higher FPR settings reduced the ML time to OHCA recognition.
Fig. 2Proportion of OHCA recognized within 30, 60 and 90 s with different false positive rate settings in the machine learning framework (all OHCA calls, n = 851). The dotted lines illustrate the proportion of OHCA recognized by the dispatcher within 30, 60 and 90 s.
Call characteristics and association with recognition rate and time to recognition
Results using univariate logistic regression showed associations between call characteristics and recognition of OHCA within 60 s by dispatcher and the ML (Table 4). When CPR was in progress, the dispatcher was 10 times more likely to recognize the OHCA than when CPR was not initiated before the emergency call (95% CI, 6.55–17.17). The corresponding results for the ML was 2 times more likely (95% CI, 1.89–4.52). ML recognition was positively associated in calls were the bystander knew the patient versus when the patient was unfamiliar (OR, 1.97; 95% CI, 1.25–3.12). When the OHCA was witnessed by a bystander, both the dispatchers and the ML were less likely to recognize the OHCA (OR, 0.42; 95% CI, 0.30–0.58) compared with incidents that were unwitnessed (OR, 0.43; 95% CI, 0.32–0.58). Recognition by the ML was negatively associated when the caller/bystander was a health care professional versus when the caller was a layperson (OR, 0.66; 95% CI, 0.48–0.91).
Table 4Associations between call characteristics and recognition of OHCA within 60 s, analysed by univariate logistic regression analyses.
n/total
Recognition by dispatcher
Recognition by machine learning
Odds ratio
95% CI
P value
Odds ratio
95% CI
P value
Bystanders
CPR in progress
95/851
10.61
6.55–17.17
<0.001
2.92
1.89–4.52
<0.001
Female
512/851
1.07
0.78–1.48
0.645
1.28
0.95–1.71
0.093
Known the patient
736/851
1.16
0.73–1.86
0.520
1.97
1.25–3.12
0.003
Health care professional
249/851
1.21
0.87–1.70
0.246
0.66
0.48–0.91
0.011
Alone on site
325/848
0.79
0.57–1.09
0.160
1.32
0.99–1.76
0.054
Witnessed OHCA
420/839
0.42
0.30–0.58
<0.001
0.43
0.32–0.58
<0.001
Dispatchers
Consciousness addressed
773/851
0.89
0.53–1.51
0.68
0.83
0.51–1.34
0.451
Breathing addressed
830/851
3.23
0.74–14.01
0.116
1.81
0.65–4.99
0.251
An odds ratio greater than one means the factor was positively associated with recognition, and odds ratios less than one mean the factor was associated with failing to recognize the OHCA.
When breathing was addressed during the call, the dispatcher was four times more likely to recognize the OHCA versus when breathing not addressed (OR, 4.15; 95% CI, 1.71–10.05). Corresponding results for the ML were similar (OR, 3.86; 95% CI, 1.56–9.53) (Supplementary Table 1).
Priority 1–4 calls (cohort B)
In the random sample of 85,205 calls, labelled as no OHCA, the ML suspected an OHCA at any time in 0.8% of the calls (n = 697). Of the ML-suspected OHCAs, a manual audit of 100 randomly selected calls showed one OHCA (also recognized by the dispatcher); in 9% (n = 9) of calls, the dispatchers also suspected an OHCA; 74% (n = 74) of calls involved an unconscious patient; and in 4% (n = 4) of calls, the dispatchers should have suspected an OHCA based on the information presented by the caller.
Discussion
The main findings are threefold. First, the ML recognized a higher proportion of OHCAs within the first minute compared with dispatchers. Second, when emergency calls with ongoing CPR were excluded, ML proved to be even more beneficial. Third, small adjustments in the FPR settings of the ML affect the time to recognition.
OHCA recognition
Even though our study used an ML with a lower FPR rate compared with Blomberg et al., the results of the present study confirm previous findings that ML, based on historical audio files, is faster and has a higher sensitivity than a dispatcher.
The ML has the potential to be a supportive tool to aid dispatchers in recognizing OHCAs during live emergency calls and thus, potentially increasing survival rates.
ML frameworks with a low FPR setting result in fewer recognized OHCAs. When the FPR setting is increased, the performance of the ML improves. In a real-life setting, a high FPR may lead to a false ML interpretation, resulting in emergency calls that do not involve an OHCA. This may result in an increased unwanted dispatch level, depleting available resources for other patients in need of EMS. There is also a risk dispatchers lose trust in the ML alerts and start ignoring its alerts in future emergency calls. One solution could be that the ML provides two types of alerts, with different FPR settings: (1) a pre-alert, set on a higher FPR setting (e.g., FPR 2.0–2.5), with advice to ensure consciousness and breathing, and (2) suspected OHCA alert, set on a lower FPR setting (e.g., FPR 1.5).
Call characteristics and recognition
Witnessed OHCA has been found to be associated with less chance of or prolonged time to recognition compared with unwitnessed cases due to the presence of agonal breathing and/or seizures.
Interaction between emergency medical dispatcher and caller in suspected out-of-hospital cardiac arrest calls with focus on agonal breathing. A review of 100 tape recordings of true cardiac arrest cases.
When the OHCA was witnessed by a bystander, both the dispatchers and the ML were less likely to recognize the OHCA compared with incidents that were unwitnessed.
Priority 1–4 calls
To evaluate its specificity, the ML was also exposed to a random sample of emergency calls labelled as non-OHCA calls (cohort B). In cohort B, the ML suspected an OHCA in 0.8% (n = 697) of 85,205 emergency calls. The manual audit of the ML-suspected OHCAs showed that the most common condition (74%) in these patients was unconsciousness. In a real live setting, a supplementary question from the dispatcher, regarding whether the victim is breathing normally would result in the dispatcher being able to reject the suspicion.
Only 1% of medical emergency calls to EMDC involve an OHCA.
Therefore, dispatchers rarely handle this type of emergency call. This might have a negative effect on dispatchers’ inclination to suspect an OHCA and their ability to recognize it. Hardeland et al. showed how education, targeted simulations and feedback on OHCA recognition improves dispatchers’ performance in recognizing OHCAs (89%–95%) and reduces delayed OHCA recognition (21% to 6%).
Education and training need to be implemented continuously in the organization to have an effect, which is time and resource consuming. A modified education and training approach in combination with the ML as a supportive tool could improve dispatchers’ performance in a similar way.
Future studies
We suggest that the optimal FPR setting of the ML needs to be evaluated in a prospective study. Dispatchers’ experience of integrating with the ML should also be examined so ML is used in an optimal way in a real-life setting.
Limitations
Dispatchers could have recognized an OHCA at an earlier time point, but not verbally communicated the need for CPR due to circumstances present at the scene.
It is unclear what clinical significance a time difference of 28–34 s, as shown in this study, has on patients survival, but it can be assumed that it may play a role in bystander CPR situations.
The ML was tested on two cohorts thus we cannot calculate negative or positive predictive values (PPV). However, with the low proportion of OHCA calls overall, even with high sensitivity and specificity, the PPV would probably be low. As mentioned earlier, a possible solution for this is a two-step process with pre-alert and OHCA alert. We are not able to report dispatchers’ specificity, because it would require considerable further manual audit of calls. At some EMDCs, the call is connected to a public safety answering point, in Sweden, all calls are handled by the dispatcher directly. Thus, it is difficult to compare the time for recognition between studies. Dispatchers trained in a different way or different dispatch systems could affect the results in different directions.
The ML framework recognized a higher proportion of OHCAs within the first minute compared with dispatchers and has the potential to be a supportive tool during live emergency calls. Optimal FPR settings need to be evaluated in a prospective study.
Funding
Corti Company funded the programming and language training phase of machine learning. Funding was received from Laerdal Foundation and Riksförbundet HjärtLung. None had any role in the design, analysis or writing of the article.
We would like to thank SOS Alarm AB for helping us with the voice log retrievals and special thanks to Jenny Hernerud for helping us with the manual audit. We also thank Corti ApS who created the ML framework. They had no role in the analysis or writing of the article.
Appendix A. Supplementary data
The following are Supplementary data to this article:
Flowchart of the study, priority 1–4 calls (cohort B).
References
Gräsner J.T.
Lefering R.
Koster R.W.
EuReCa ONE-27 Nations, ONE Europe, ONE Registry: a prospective one month analysis of out-of-hospital cardiac arrest outcomes in 27 countries in Europe.
Different impacts of time from collapse to first cardiopulmonary resuscitation on outcomes after witnessed out-of-hospital cardiac arrest in adults, circulation: cardiovascular quality and outcomes.
Cardiac arrest and cardiopulmonary resuscitation outcome reports: update and simplification of the Utstein templates for resuscitation registries. A statement for healthcare professionals from a task force of the International Liaison Committee on Resuscitation (American Heart Association, European Resuscitation Council, Australian Resuscitation Council, New Zealand Resuscitation Council.
Interaction between emergency medical dispatcher and caller in suspected out-of-hospital cardiac arrest calls with focus on agonal breathing. A review of 100 tape recordings of true cardiac arrest cases.