Presenting Author:

Celia O'Brien, Ph.D.

Principal Investigator:

Celia O'Brien, Ph.D.

Department:

Medical Education

Keywords:

undergraduate medical education, assessment, competency, portfolio, qualitative, narrative feedback

Location:

Third Floor, Feinberg Pavilion, Northwestern Memorial Hospital

E9 - Education

What problematic behaviors persist across time? A qualitative study of narrative feedback to medical students

Our medical school utilizes a portfolio to assess behavioral competencies such as communication, professionalism and teamwork. At the end of the pre-clerkship phase of our curriculum, a faculty committee reviews portfolios to detect themes in performance and identify students with concerns in the above domains. In the 2014 review, faculty identified 20% of students (31/156) with patterns of concerning behavior that could impact clerkship performance. Quantitative analyses of the same students conducted at the end of core clerkships showed that the concerning behavior(s) identified in the pre-clerkship review negatively predicted clerkship grades, even after controlling for academic ability. These results suggest that the portfolio is identifying behaviors that persist into clerkships, but we do not know which of these behaviors are negatively impacting performance. This qualitative study seeks to understand the specific types of longitudinal behaviors that impact performance in medical school and to discover which problematic aspects of communication, professionalism and teamwork skills are persistent over time. Methods The sample consists of 48 students: 24 who had pre-clerkship concerns and 24 randomly selected students who did not. We used qualitative content analysis in three stages. First, two researchers coded narrative feedback from the pre-clerkship portfolio reviewers and clerkship directors. The coders identified feedback on behaviors and whether it was positive or negative. The coders created a common coding structure which was refined by a third reviewer. To reduce bias, coders were blinded to the results of the portfolio review and analyzed pre-clerkship and clerkship feedback separately. In the second stage, we examined whether the same behavior code appeared across both phases for individual students. In the third stage, if the same behavior was coded in both phases, two independent researchers read both sets of feedback, decided whether they described the same behavior, and met to reconcile differences. Types of consistent behavior were then compared between student groups (concerns vs. no concerns). Results Students with pre-clerkship concerns were more likely to receive negative feedback on the same behaviors across time. In particular, they received more negative feedback on demeanor, communication with colleagues and patient rapport. Specific behaviors that persisted across time were demeanors that projected disinterest or disengagement, non-participation in colleague discussions and difficulty establishing connections with patients. Discussion The results of this study identify specific problematic behaviors that are persistent in medical students. Our faculty reviewers were able to identify behavioral patterns that were later verified by clerkship directors. Earlier identification of these issues is essential to facilitate remediation and to help students reach their full potential and deliver high-quality patient care.