University of Wisconsin–Madison ⋅ Medical College of Wisconsin
WMJ

Simulation-Based Medical Education: Development of an Assessment Tool for Novice Use

Sasha M. Ulrich, MD; Joseph C. L’Huillier, MD; Sarah A. Jung, PhD; Laura K. Krecko, MD; Alexandra A. Rosser, BS; Amy K. Schulze, MD; Amy E. Liepert, MD; Ann P. O’Rourke, MD, MPH

WMJ. 2022;121(4):316-322

Download full text pdf. Download Appendices.

ABSTRACT

Background: Simulation-based medical education, an educational model in which students engage in simulated patient scenarios, improves performance. However, assessment tools including the Oxford Non-Technical Skills (NOTECHS) scale require expert assessors. We modified this tool for novice use.

Methods: Medical students participated in 5 nontechnical simulations. The NOTECHS scale was modified to allow for novice evaluation. Three novices and 2 experts assessed performance, with intraclass correlation used to assess validity.

Results: Twenty-two learners participated in the simulations. Novice reviewers had moderate to excellent correlation among evaluations (0.66 < intraclass correlation coefficients [ICC] < 0.95). Novice and expert reviewers had moderate to good correlation among evaluations (0.51 < ICC < 0.88).

Discussion: The modified NOTECHS scales can be utilized by novices to evaluate simulation performance. Novice assessment correlates with expert review. These tools may encourage the use of simulation-based medical education.

Author Affiliations: Mayo Clinic Health System, Department of Family Medicine, Eau Claire, Wisconsin (Ulrich); University of Wisconsin School of Medicine and Public Health, Department of Surgery, Madison, Wis (Ulrich, L’Huillier, Jung, Krecko, Rosser, O’Rourke); University at Buffalo, Jacobs School of Medicine and Biomedical Sciences, Department of Surgery, Buffalo, New York (L’Huillier, Schulze); University of California San Diego Health, Department of Surgery, San Diego, California (Liepert).
Corresponding Author: Ann P. O’Rourke, MD, MPH, University of Wisconsin School of Medicine and Public Health, Department of Surgery, Clinical Science Center, 600 Highland Ave, Madison, WI 53792; phone 608.262.6246; email orourke@surgery.wisc.edu.
Funding/Support: This work was supported by the National Cancer Institute of the National Institutes of Health (NIH), grant T32CA090217. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The authors would like to thank the University of Wisconsin School of Medicine and Public Health Shapiro Summer Research Fellowship Program and the Department of Surgery for their financial support of this project.
Financial Disclosures: None declared.
Share WMJ