Mastery Learning Standards for Central Line: A Patient Safety Approach

Background: Mastery learning is an instructional approach in which learners advance based on skill acquisition rather than curricular time. It can be used for simulation-based education when patient-safety considerations are key, like learning procedural skills on a simulator before performing it on a patient. Setting standards is a key component of mastery-learning programs, determining when learners ready to move on to the next skill or task. Traditional standard setting methods require modifications for a mastery-learning environment. We sought to compare the results of traditional Angoff and Hofstee standard setting for central venous catheter (CVC) insertion skills with those of methods explicitly taking mastery learning and patient safety approaches. Methods: Twelve physicians experienced in CVC insertion set cut-scores for internal jugular (IJ) and subclavian (SC) CVC insertion checklists using four standard setting approaches. Group-level performance on each item before and after CVC training was provided to inform judges. The methods were: 1) Traditional Angoff, indicating the probability that a “borderline” or minimally competent resident would accomplish each item; 2) Hofstee indicating the minimum and maximum acceptable passing scores and failure rates; 3) Mastery learning Angoff, indicating the probability that a “well prepared” resident would accomplish each item; and 4) Patient-safety standards based on identifying critical and non-critical items. The resulting cut-scores were compared using historical performance of internal and emergency medicine residents who participated in a simulation-based mastery learning curriculum for CVC insertion. Results: The IJ and SC CVC checklists each included 29 items. Faculty deemed all items on the IJ and SC checklists as “critical” to patient safety, patient comfort, and/or procedure outcome; two items were deemed critical for patient comfort only. Final cut scores were: Traditional Angoff: IJ 91%, SC 90%; Hofstee: IJ 88%, SC 90%; Mastery learning Angoff IJ 98%, SC 98%; Patient safety IJ 98%, SC 98%. A cut score of 88% required accomplishing 25/29 items; 90-91% required 26/29 items, and 98% required 28/29 items. Based on the historical performance of 92 residents assessed on IJ and SC insertion applying the 98% standard would result in requiring additional practice and retest of 36/81 residents (44%) who had passed the IJ checklist and 35/84 residents (42%) who had passed the SC checklist using traditional standards. Conclusion: The mastery learning and patient safety approaches to standard setting for CVC insertion checklists resulted in cut scores that required residents to demonstrate a higher level of proficiency before performing the procedure on live patients than did traditional standard setting methods. Faculty responsible for setting standards for mastery learning of procedural skills should implement appropriate methods to address mastery learning and patient safety considerations.