Abstract

NM-374

Cumulative sum (CUSUM) analysis: A novel tool to assess learning of ultrasound-guided vascular access procedures among pediatric anesthesiology trainees

O'Malley B, Careskey M, Hernandez C, Campion M, Mecoli M
Cincinnati Children's Hospital, Cincinnati, OH, USA

Introduction: Vascular access is a critical but often challenging component in providing safe anesthetic care to pediatric patients, and ultrasound (US) guidance is increasingly utilized in cases of particularly difficult access. Similar to other procedures in anesthesia, it is suspected that proficiency in US-guided establishment of vascular access among learners should improve over time, and that mastery of skill should be expected after a certain number of attempts.
CUSUM analysis is a well-described statistical model that quantifies an individual’s progress over time as they learn a new procedural skill (1). CUSUM has been used to determine proficiency in various clinical procedures based on establishment of acceptable and unacceptable failure rates (1). Here, we describe CUSUM analysis as an assessment tool to evaluate acquisition and mastery of US-guided vascular access procedures in our current class of pediatric anesthesia fellows.
Methods: Thirteen pediatric anesthesia fellows are self-reporting their successful and unsuccessful attempts at US-guided peripheral, central venous, and arterial catheter placement via a web-based application on their mobile phones; each needle insertion through the skin is counted as an attempt. Individual CUSUM plots were created for each fellow based on procedure type. Acceptable and unacceptable failure rates of 25% and 50% respectively were established prior to data collection based on unpublished reports from experienced faculty in our department. Skill acquisition is considered successful when the graphical trend falls below the CUSUM value representative of an acceptable failure rate.
Results: CUSUM analysis shows that learners currently fall into three categories for each procedure – skill achieved, skill not yet achieved, and insufficient data for interpretation – with representative curves shown in Figure 1A. The timeframe for acquisition of skill varies among learners and depends on the procedure subtype. Complete skill acquisition results for each learner are compiled in Figure 1B.
Discussion: Our data demonstrate the utility of CUSUM in determining competency for various US-guided vascular access procedures. There is varying skill acquisition for these procedures, with some learners achieving skill competency faster than others. Data suggest that US-guided PIV placement might be a more difficult skill to master than central and arterial lines. Limitations of the CUSUM methodology include the need for self-reporting and large number of attempts required to prove statistical significance. Further data are needed to better define acceptable failure rates among learners, and to understand reasons for variance in number of trials leading to skill acquisition.
Conclusion: CUSUM analysis can be used as an evaluation tool for competence in US-guided vascular access procedures among learners of pediatric anesthesia.
References:
1. Starkie, T, et al. Can J Anesth. 2013. 60: 1228–1239