Rating medical emergency teamwork performance: Development of the Team Emergency Assessment Measure (TEAM)



      To develop a valid, reliable and feasible teamwork assessment measure for emergency resuscitation team performance.


      Generic and profession specific team performance assessment measures are available (e.g. anaesthetics) but there are no specific measures for the assessment of emergency resuscitation team performance.


      (1) An extensive review of the literature for teamwork instruments, and (2) development of a draft instrument with an expert clinical team. (3) Review by an international team of seven independent experts for face and content validity. (4) Instrument testing on 56 video-recorded hospital and simulated resuscitation events for construct, consistency, concurrent validity and reliability and (5) a final set of ratings for feasibility on fifteen simulated ‘real time’ events.


      Following expert review, selected items were found to have a high total content validity index of 0.96. A single ‘teamwork’ construct was identified with an internal consistency of 0.89. Correlation between the total item score and global rating (rho 0.95; p < 0.01) indicated concurrent validity. Inter-rater (k 0.55) and retest reliability (k 0.53) were ‘fair’, with positive feasibility ratings following ‘real time’ testing. The final 12 item (11 specific and 1 global rating) are rated using a five-point scale and cover three categories leadership, teamwork and task management.


      In this primary study TEAM was found to be a valid and reliable instrument and should be a useful addition to clinicians’ tool set for the measurement of teamwork during medical emergencies. Further evaluation of the instrument is warranted to fully determine its psychometric properties.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Resuscitation
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Cooper S.
        • Janghorbani M.
        • Cooper G.
        A decade of in-hospital resuscitation: outcomes and prediction of survival.
        Resuscitation. 2006; 68: 231-237
        • Abella B.S.
        • Alvarado J.
        • Myklebust H.
        • et al.
        Quality of cardiopulmonary resuscitation during in-hospital cardiac arrest.
        JAMA. 2005; 293: 305-310
        • Cooper S.
        • O’Carroll J.
        • Jenkin A.
        • Badger B.
        Collaborative practices in unscheduled emergency care: role and impact of the emergency care practitioner qualitative and summative findings.
        Emerg Med J. 2007; 24: 625-629
        • DeVita M.A.
        • Schaefer J.
        • Lutz J.
        • Wang H.
        • Dongilli T.
        Improving medical emergency team (MET) performance using a novel curriculum and a computerized human patient simulator.
        Qual Saf Health Care. 2005; 14: 326-331
        • Thomas E.J.
        • Taggart B.
        • Crandell S.
        • et al.
        Teaching teamwork during the Neonatal Resuscitation Program: a randomized trial.
        J. Perinatol. 2007; 27: 409-414
        • Ummenhofer W.
        • Amsler F.
        • Sutter P.
        • Martina B.
        • Martin J.
        • Scheidegger D.
        Team performance in the emergency room: assessment of inter-disciplinary attitudes.
        Resuscitation. 2001; 49: 39-46
        • Flin R.
        • Maran N.
        Non-technical skills: identifying, training, and assessing safe behaviours.
        in: Riley R. Manual of simulation in healthcare. Oxford University Press, 2008
        • Fletcher G.
        • Flin R.
        • McGeorge P.
        • Glavin R.
        • Maran N.
        • Patey R.
        Anaesthetists’ non-technical skills (ANTS): evaluation of a behavioural marker system.
        Br J Anaesth. 2003; 90: 580-588
        • Flin R.
        • Yule S.
        Leadership for safety: industrial experience.
        Qual Saf Health Care. 2004; 13
        • Flin R.
        • Maran N.
        Identifying and training non-technical skills for teams in acute medicine.
        Qual Saf Health Care. 2004; 13
      1. Endsley MR. Situation awareness global assessment technique (SAGAT). National Aerospace and Electronics Conference (NAECON). New York, 1988; p. 798–95.

      2. Health and Safety Executive. Health and safety Executive; 2009. [accessed 19.08.09].

        • Wayne D.B.
        • Butter J.
        • Siddall V.J.
        • et al.
        Simulation-based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial.
        Teach Learn Med. 2005; 17: 210-216
        • Clark P.
        Teamwork: building healthier workplaces and providing safer patient care.
        Crit Care Nurs Quart. 2009; 32: 221-231
        • Perkins G.D.
        • Boyle W.
        • Bridgestock H.
        • et al.
        Quality of CPR during advanced resuscitation training.
        Resuscitation. 2008; 77: 69-74
      3. Resuscitation Council. Immediate Life Support Resuscitation Council; 2006.

        • Kim J.
        • Neilipovitz D.
        • Cardinal P.
        • Chiu M.
        • Clinch J.
        A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: the University of Ottawa critical care medicine, high-fidelity simulation, and crisis resource management I study.
        Crit Care Med. 2006; 34: 2167-2174
        • Small S.D.
        • Small S.D.
        Simulation applications for human factors and systems evaluation.
        Anesthesiol Clin. 2007; 25: 237-259
        • Brown D.
        • Chronister C.
        The effect of simulation learning on critical thinking and self-confidence when incorporated into an electrocardiogram nursing course.
        Clin Simul Nurs. 2009; 5: 45-52
        • Moorthy K.
        • Munz Y.
        • Adams S.
        • Pandey V.
        • Darzi A.
        A human factors analysis of technical and team skills among surgical trainees during procedural simulations in a simulated operating theatre.
        Ann Surg. 2005; 242: 631-639
        • Marsch S.C.U.
        • Müller C.
        • Marquardt K.
        • Conrad G.
        • Tschan F.
        • Hunziker P.R.
        Human factors affect the quality of cardiopulmonary resuscitation in simulated cardiac arrests.
        Resuscitation. 2004; 60: 51-56
        • National Patient Safety Association
        Team Climate Awareness Measure (TCAM).
        National Patient Safety Association, 2005
        • Manser T.
        Teamwork and patient safety in dynamic domains of healthcare: a review of the literature.
        Acta Anaesthesiol Scand. 2009; 53: 143-151
        • Yule S.
        • Flin R.
        • Paterson-Brown S.
        • Maran N.
        Non-technical skills for surgeons in the operating room: a review of the literature.
        Surgery. 2006; 139: 140-149
        • Cooper S.
        • Wakelam A.
        Leadership of resuscitation teams: ‘Lighthouse Leadership’.
        Resuscitation. 1999; 42: 27-45
        • Malec J.
        • Torsher L.
        • Dunn W.
        • et al.
        The Mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills.
        Simul Healthc. 2007; 2: 4-10
        • Cooper S.
        • O’Carroll J.
        • Jenkin A.
        • Badger B.
        Collaborative practices in unscheduled emergency care: role and impact of the emergency care practitioner—quantitative findings.
        Emerg Med J. 2007; 24: 630-633
        • Dieckmann P.
        • Molin Friis S.
        • Lippert A.
        • Ostergaard D.
        The art and science of debriefing in simulation: ideal and practice.
        Med Teach. 2009; 31: 287-294
        • Cox K.
        No oscar for OSCE.
        Med Educ. 1990; 24: 540-545
        • Regehr G.
        • Macrae H.
        • Reznick R.
        • Szalay D.
        Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE format examination.
        Acad Med. 1998; 73: 993-997
        • Polit D.
        • Beck C.
        The content validity index: are you sure you know what's being reported? Critique and recommendations.
        Res Nurs Health. 2006; 29: 489-497
        • Worster A.
        • Haines T.
        Advanced statistics: understanding medical record review (MRR) studies.
        Acad Emerg Med. 2004; 11: 187-192
        • Fliess J.L.
        Statistical Methods for Rates and Proportions.
        Wiley, New York1981
        • Gravetter F.J.
        • Wallnau L.B.
        Statistics for the behavioural sciences.
        3rd ed. West Publishing Company, New York1992
      4. Resuscitation Council United Kingdom. Advanced life support manual. 4th ed.; 2000.

        • Reader T.
        • Flin R.
        • Lauche K.
        • Cuthbertson B.
        Non-technical skills in the intensive care unit.
        Br J Anaesth. 2006; 96: 551-559
        • Niebuhr R.E.
        • Manz C.C.
        • Davis K.R.
        Using video tape technology: innovations in behavioural research.
        J Manage. 1981; 7: 43-54
        • Reise S.
        • Waller N.
        • Comrey A.
        Factor analysis and scale revision.
        Psychol Assess. 2000; 12: 287-297
        • Bond S.
        • Cooper S.
        Modelling emergency decisions: recognition-primed decision making. The literature in relation to an ophthalmic critical incident.
        J Clin Nurs. 2006; 15: 1023-1032
        • Klein G.
        The power of intuition. How to use your gut feelings to make better decisions at work.
        Doubleday, New York2003
        • Yule S.
        • Flin R.
        • Maran N.
        • Rowley D.
        • Youngson G.
        • Paterson-Brown S.
        Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system.
        World J Surg. 2008; 32: 548-556
        • Holcomb J.B.
        • Dumire R.D.
        • Crommett J.W.
        • et al.
        Evaluation of trauma team performance using an advanced human patient simulator for resuscitation training.
        J Trauma Inj Infect Crit Care. 2002; 52 (discussion 85–6): 1078-1085
        • Senior B.
        • Swailes S.
        Inside management teams: developing a teamwork survey instrument.
        Br J Manage. 2007; 18: 138-153
        • Mickan S.M.
        • Rodger S.A.
        Effective Health Care Teams: a model of six characteristics developed from shared perceptions.
        J Interprof Care. 2005; 19: 358-370
        • Flin R.
        • Martin L.
        • Goeters K.
        • et al.
        Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills.
        Human Factors Aerospace Saf. 2003; 3: 97-120
        • Ottestad E.
        • Boulet J.
        • Lighthall G.
        Evaluating the management of septic shock using patient simulation.
        Crit Care Med. 2007; 35: 769-775
        • Morgan P.
        • Pittini R.
        • Regehr G.
        • Marrs C.
        • Haley M.
        Evaluating teamwork in a simulated obstetric environment.
        Anesthesiology. 2007; 106: 907-915
        • Mitchell L.
        • Flin R.
        Non-technical skills of the operating theatre scrub nurse: literature review.
        J Adv Nurs. 2008; 63: 15-24