Let’s Choose Residents Differently


Click beneath for the audio version of the blog entry:



Many Are Wondering How To Choose Residents

One of the ways we identify new blog topics is by searching the web and reading social media with content from our colleagues.  Sometimes, this overlaps with our area of expertise and things in which we at the blog have an interest.  For about the last 5 years I have had a strong interest in how resident surgeons are selected by training programs.  Recently, one of my co-bloggers ran across an entry from one of our colleagues that also referenced this difficult topic.  What are some of the ways to select for resident surgeons among applicants so we can identify who will eventually make excellent attending surgeons?  How should we be selecting our residents?


Here’s What Doesn’t Work

First, let’s consider what doesn’t work in resident selection.  Here we find most of the criteria that are routinely used to select residents.  Things such as USMLE scores and other classically used selection criteria just plain don’t work.  There is significant evidence for this and let’s avoid reciting what my colleagues at other blogs have described.  We will simply point you to them with a link here.


Why Don’t We Use Things That Work?

Next, let’s consider some potential reasons why the criteria we use frequently don’t work and why we choose to try them.  For one, typically, we as surgeons are not educated in how to select people for different jobs.  We may think one good surgeon can tell someone else who can be a good surgeon by a combination of personal interviewing, test scores, or some other routine criteria.  However, as my colleague the Skeptical Scalpel points out very adeptly, that doesn’t seem to be the case.


In short, we are not educated on what selection criteria can be utilized to find people who will be effective in their profession. Often, attending surgeons who help to select residents have little to no education in the significant amount of human resource literature that has been created on how to select people for certain positions.  Just like many things in life, there may be some science and literature that makes things a little easier.  Let’s take a look.


First, consider that, as a surgeon, I would not operate unless I had been educated in how to perform a procedure with respect to the technical aspects of the procedure as well as the cognitive ones.  However, I find that (as physicians) we are typically asked to perform tasks for which we have no education.  This is often because physicians are bright, articulate, and dedicated.  However, being bright, dedicated, and articulate alone is not enough to fly a plane, play a violin, or perform any other complex task for which there are skills and cognitive dimensions required in which we need education.  Just as we do not ask people who are non-surgeons to understand all of the surgical literature, so it is that we as physicians are often left to select new and up-coming surgeons without being aware of the significant body of knowledge that exists on how to select people for different positions–in essence we are asked to do something that we aren’t trained to do.  In short, we have no education about the significant evidence on how to select people and yet we are asked to fly the plane based on intuition alone.


Allow me (unsupported by anything but experience) to say that, as confident surgeons we typically don’t know or don’t seek out all of this relevant data in part because we make so many other high stakes, challenging decisions.  This selection can be seen as just one more and it may feel nowhere near the most important decision we will make in a day in that the repercussions are not immediately felt.  (Of course, it is perhaps one of the most important decisions we will make in a day in that it will affect many people in years to come.) For that reason, I believe, we don’t seek out literature on how to select people.


There’s Literature About How To Select

Here is the moment we push the soapbox aside.  Today, let me share just a few of the relevant data points on selecting people (and getting a sense for eventual job performance) from various organizational behavior and psychology entries into the literature.  I first saw these in an MBA course and was impressed about the breadth and depth of literature there is behind how to select people effectively for different roles.  It impressed me that there was such a body of knowledge and that I had previously been ignorant of it.  (Guess that’s how I knew the education was working.) After, we will describe some of the typically used selection criteria other than the more trivial ones listed above.  such as medical school grades, USMLE scores, and letters of recommendation to name a few.  As mentioned, my colleagues on other blogs have clearly highlighted issues with these and shown their shortcomings.  We will then describe some of the other typically used selection criteria as well as their potential utility for selecting people who will make effective residents and, more importantly, effective attending surgeons.


You’ve Heard Of The MBTI, And It’s Not The Answer

First, let’s consider the Myers-Briggs Type Indicator or MBTI.  This is perhaps the most widely used personality assessment instrument in the world, and it’s likely you’ve heard of it.  The MBTI is a hundred question personality test that focuses on how people feel in certain venues.  People become classified as either Extroverted or Introverted (E or I), Sensing or Intuitive (S or N), Thinking or Feeling (T or F) and Judging or Perceiving (J or P).  Each of these has a specific definition.  Based on this, a four letter type is then generated for each person with 16 potential personality types.  The MBTI is widely used by multiple organizations including the US Armed Forces, Apple Computer, AT&T, and GE to name some of the larger companies.


However, most evidence suggests the MBTI is not a valid measure of personality and it appears as though the MBTI results tend to be unrelated to job performance.  Thus, selecting people based on Myers-Briggs Type Indication probably does not effectively predict their job performance.  See, for instance, RM Capraro and MM Capraro’s article “Myers-Briggs Type Indicator score reliability across studies: A meta-analytic reliability generalization study in education and psychological measurement”, August 2002, page 590-602.  Also, see the RC Arnau et al article “Are Jungian preferences really categorical? An empirical investigation using taxometric analysis in Personality and Individual Differences”, January 2003 page 233-251 for additional information.  So, dividing people into these classic personality types does not seem to be useful in predicting job performance.  Are there any reliable indicators that can help us in order to select people for different positions?  Let me tell you what I have learned, some of the evidence, and the factor I use to help select young physicians for residency roles and beyond.


The Big 5 Can Help

By way of contrast to the MBTI, the Big 5 model (also called the Big 5) has a significant amount of supporting evidence that indicates 5 basic dimensions are responsible for significant variation in human personality.  This is paraphrased from Robins and Judge, Essentials of Organizational Behavior 9th Edition and much of the section above we have just paraphrased comes from that text.  The Big 5 personality model delineates 5 dimensions of personality including: Extroversion, Agreeableness, Conscientiousness, Stability and Openness to experience.  A significant amount of research indicates that those 5 basic personality factors underlie all other types.  It turns out that research on the Big 5 has found relationships between these personality dimensions and job performance.


Conscientiousness Predicts Performance Across A Wide Range of Occupations

Researchers examined a broad spectrum of occupations including engineers, architects, accountants, attorneys, police, managers, sales people, and both semi-skilled and skilled employees.  Results indicated that across all occupation groups, conscientiousness predicted job performance.  For more information including the primary work see J Hoogen and B Holland “Using Theory to Evaluate Personality and Job Performance relations: A socioanalytic perspective” Journal of Applied Psychology, February 2003, page 100-112.  Also MR Barrick and MK Mount “Select on conscientiousness and emotional stability” in EA Lock’s The Handbook of Principles of Organizational Behavior 2004, page 15-28.


Since I first learned about the Big 5 model I have attempted to assess conscientiousness as best I can as the most important predictor of success for prospective residents.  Yes, I do not administer a personality test; however, I do my best to get at the dimension of conscientiousness during job interviews and other interactions.  Experientially, this seems to work.  Yes, there are many types of interviewing styles which have also been delineated.  However, I find that the Big 5 personality inventory’s dimension of conscientiousness, assessed as best as possible, does seem to indicate the level of function for the eventual resident and beyond.


Do Something (Anything) Besides What We’ve Done So Far

Clearly, there are multiple potential manners in which we can select residents.  I think that it is worthwhile that we go on the hunt for new selection criteria owing to the fact that we are acutely aware of the limitations in the ones we typically use.  As mentioned, there is an extensive body of research that exists about how to select people for different roles and we’ve highlighted some here.  The unfortunate fact is that many of us are simply not educated in the significant body of knowledge that exists around how to select people for roles owing to the fact that we have no exposure to this in our training, are often busy working, and make other high-stakes decisions which gives us a certain comfort level with making hiring decisions.  My recommendation is that we take a moment to reconsider how we select for residents and turn towards more evidence-based models such as the Big 5 personality inventory or at least the dimension of conscientiousness assessed as best we can.


Disagree?  Comments, thoughts or discussion?  Please share your thoughts beneath.