Resuscitation
Volume 81, Issue 3 , Pages 331-336, March 2010

Reliability and validity of a scoring instrument for clinical performance during Pediatric Advanced Life Support simulation scenarios☆

  • Aaron Donoghue

      Affiliations

    • Division of Pediatric Critical Care Medicine, Children's Hospital of Philadelphia, PA, United States
    • Division of Pediatric Emergency Medicine, Children's Hospital of Philadelphia, PA, United States
    • Children's Hospital of Philadelphia, Center for Simulation, Advanced Education, and Innovation, Philadelphia, PA, United States
    • Corresponding Author InformationCorresponding author at: Division of Emergency Medicine, Children's Hospital of Philadelphia, 34th Street and Civic Center Boulevard, Philadelphia, PA 19104, United States. Tel.: +1 215 590 1289; fax: +1 215 590 4454.
    • ,
  • Akira Nishisaki

      Affiliations

    • Division of Pediatric Critical Care Medicine, Children's Hospital of Philadelphia, PA, United States
    • Children's Hospital of Philadelphia, Center for Simulation, Advanced Education, and Innovation, Philadelphia, PA, United States
    • ,
  • Robert Sutton

      Affiliations

    • Division of Pediatric Critical Care Medicine, Children's Hospital of Philadelphia, PA, United States
    • Children's Hospital of Philadelphia, Center for Simulation, Advanced Education, and Innovation, Philadelphia, PA, United States
    • ,
  • Roberta Hales

      Affiliations

    • Children's Hospital of Philadelphia, Center for Simulation, Advanced Education, and Innovation, Philadelphia, PA, United States
    • ,
  • John Boulet

      Affiliations

    • Foundation for Advancement of International Medical Education and Research, Philadelphia, PA, United States

Received 26 July 2009; received in revised form 10 October 2009; accepted 23 November 2009. published online 04 January 2010.

Abstract 

Aim

To assess the reliability and validity of scoring instruments designed to measure clinical performance during simulated resuscitations requiring the use of Pediatric Advanced Life Support (PALS) algorithms.

Methods

Pediatric residents were invited to participate in an educational trial involving simulated resuscitations that employ PALS algorithms. Each subject participated in a session comprised of four scenarios (asystole, dysrhythmia, respiratory arrest, shock). Video-recorded sessions were independently reviewed and scored by four raters using instruments designed to measure performance in terms of timing, sequence, and quality. Validity was assessed by two-factor analysis of variance with postgraduate year (PGY-1 versus PGY-2) as an independent variable. Reliability was assessed by calculation of overall interrater reliability (IRR) as well as a generalizability study to estimate variance components of individual measurement facets (scenarios, raters) and associated interactions.

Results

20 subjects were scored by four raters. Based on a two-factor ANOVA, PGY-2s outperformed PGY-1s (p<0.05); significant differences in difficulty existed between the four scenarios, with dysrhythmia scores being the lowest. Overall IRR was high (0.81) and most variance could be attributed to subject (17%), scenario (13%), and the interaction between subject and scenario (52%); variance attributable to rater was minimal (1.4%).

Conclusions

The instruments assessed in this study measure clinical performance during PALS scenarios in a reliable and valid manner. Measurement error could be minimized further through the use of additional scenarios but additional raters, for a given scenario, would not improve reliability. Further studies should assess validity of measurement with respect to actual clinical performance during resuscitations.

Keywords: Simulation, Pediatric Advanced Life Support, Education

To access this article, please choose from the options below

Login to an existing account or Register a new account.

  • Purchase this article for 31.50 USD (You must login/register to purchase this article)

    Online access for 24 hours. The PDF version can be downloaded as your permanent record.

  • Subscribe to this title

    Get unlimited online access to this article and all other articles in this title 24/7 for one year.

  • Claim access now

    For current subscribers with Society Membership or Account Number.

  • Visit SciVerse ScienceDirect to see if you have access via your institution.
 

☆ “A Spanish translated version of the abstract of this article appears as Appendix in the final online version at doi:10.1016/j.resuscitation.2009.11.011”.

PII: S0300-9572(09)00592-9

doi:10.1016/j.resuscitation.2009.11.011

Resuscitation
Volume 81, Issue 3 , Pages 331-336, March 2010

Article Tools

* Image Usage & Resolution