Modification of a children's hospital pediatric early warning score (EWS): An evaluation of inter-rater reliability, nurses' critical thinking and perceptions of the tool

Pediatric early warning scores (EWS) have been utilized to assist the identification of children at risk for clinically decompensating, experiencing a cardiac or respiratory arrest, or requiring a transfer to a higher level of care. Although their use is widespread, little consistency exists between tools and research evaluating the effectiveness of these tools is lacking. This quasi-experimental project evaluated twenty-five medical-surgical staff nurses' use and perceptions as well as the inter-rater reliability of a newly modified pediatric EWS tool at a free standing, academic Midwestern pediatric hospital. The tool was modified utilizing existing literature and an interdisciplinary team's expertise. Five fictionalized patients, presented in case studies, were developed and nurses were asked to score these patients using the newly modified tool with rationale. Inter-rater reliability was assessed utilizing Fleiss' Kappa and qualitative questionnaire data was analyzed for emerging themes. Overall, Fleiss' Kappa showed that there was moderate agreement between the nurses' judgments and scoring, with scores primarily differing due to the difficulty level of each case study. Nurses' responses to a questionnaire indicated differing levels of comfort identifying and managing children that present with mid-range total scores as opposed to those who scored in the lower or higher ranges. This project's findings highlight nurses' concerns that an objective tool may not accurately describe a subjective assessment. The results of this project indicated that use of this tool, with some modifications to address nursing concerns, may help to identify clinically decompensating pediatric patients being treated on medical-surgical units.