Dr John Sullivan Talent Management Thought Leadership

You may lose up to 40% of your top candidates when you use flawed candidate assessment tools. You might be surprised to learn that many managers worldwide routinely use assessment approaches that, even on the surface, appear silly.

For example, Japanese hiring managers have been known to screen out candidates based on their blood type (O is best / AB can be a dealbreaker). Throughout China and India, astrology is used to screen out candidates whose horoscopes don’t match that of the company (in fact, an astrologist may be on their HR team). Unfortunately, in the US, managers also often use personality type classification tests like the Myers-Briggs even though the owners of the test recommend that it never be used for hiring.

Managers Need To Learn To Use Objective Criteria To Select Each Assessment Tool

So, instead of relying on intuition and what they have always used. Hiring managers and recruiters must begin making their selection of candidate assessment tools based on data. The first step in that transition is for all corporate recruiting leaders to proactively discourage using candidate assessment tools that don’t meet the following four minimum requirements for use.

• It predicts performance (validation) – those who score high on the assessment test/tool will perform above average on the job if they get it. 
• It is job-related – the tool measures a knowledge, skill, or other factor required to do the job.
• It is reliable – the tool produces consistent results among assessors. And it produces the same results if the tool is used again on the same candidate. 
• Everything is measurable – each of the assessed factors can be clearly defined, and each one can be accurately measured. 

———–——————

The 5 Worst Candidate Assessment Tools… That You Should Never Use

Over the last few years, numerous elements of the recruiting process have become obsolete. I outlined them in an article in 2021. However, this current piece has a much narrower focus. In it, I am merely trying to remind the reader about the five absolutely “worst” candidate assessment approaches that could hurt your hiring results. The “worst” assessment tools, both widely used and among the most damaging, are listed first.

The #1 worst tool – The Myers-Briggs personality test, designed as a parenting tool (MBTI) – although many still use this as part of their candidate assessment process. In fact, it is so inappropriate for hiring. The company that owns the test specifically warns those who purchase it against using it for hiring and selection. The primary flaw of this “worst” tool is that it doesn’t accurately predict job success. However, it shares another major flaw with every other personality type indicator. And that flaw is that you must first identify the “ideal personality type” that will lead to high performance in your job before determining if each candidate accurately matches your ideal personality. However, because of the expense and the time involved. No major corporation has ever publicly revealed that they have proven which specific personality type is essential for success in each job family in their organization. So, until you have data that reveals the required personality type, I recommend never using any personality type test for hiring. This warning is especially important for managers who work within small businesses. Because with a relatively small HR function, no one may warn them and prevent the actual use of Myers-Briggs.

The #2 worst tool – Job fit… may be the most subjective and discriminatory assessment tool – perhaps the most widely used of all candidate screening tools. It also has the highest number of flaws. First, you must identify the required “job fit” factors (i.e., values, loyalty, energy) that are essential for job/team success. Next, each of the identified fit factors must be clearly defined and easy to measure by untrained interviewers. Unfortunately, many fit factors relate to a candidate’s intentions and values. Accurately identifying, defining, and consistently measuring the fit factors turns out to be almost impossible. Once you do some research, you will likely find that the data shows that having too many team members with the same fit will actually cause many problems, including groupthink, lower levels of risk-taking/innovation, and a reduction in team diversity. You will also learn that many new hires succeed without being a fit because they learn how to adapt and change their behaviors and attitudes. Learn more about the many problems associated with assessing fit here.

Tool #3 – Body language… where assessment results are widely inconsistent among assessors – unfortunately, it’s extremely common for individual interviewers to attempt to assess a candidate’s body language. Even though there is little data showing that the average interviewer can accurately spot the individual body movements that many presume actually reveal the hidden behaviors, attitudes, and values of a future employee. Other flaws of the fit assessment process include the fact that interviewers from different cultures and regions will assess the same body movement differently. And the consistent assessment of body movements won’t be possible until every evaluator agrees on which body movements are most important. And, of course, what each individual movement actually reveals. That agreement among assessors will be almost impossible to get. Of course, there is no data that proves that a particular body posture or movement actually represents a future employee behavior. Taken together, these uncertainties mean that the body movement assessments between different interviewers will always vary by a wide margin (even though they observed the same interview). Add to these many reliability issues the fact that the determination of what is a proper body language message during an interview is based on Western influences. The overall assessment of body language often turns out to be highly discriminatory, especially among disabled candidates.

Tool #4 – The measurement of Emotional Intelligence (EQ) is extremely unreliable – most everyone has heard about corporate executives who are highly successful but who are clearly not always emotionally stable (Elon Musk and Donald Trump both come to mind). So, most managers are not surprised to learn that it’s hard to find credible evidence to prove you must have a certain level of emotional intelligence in order to do a particular job. The next serious flaw in this tool may be that it can be highly discriminatory. Because the very definition of what is an acceptable EQ primarily reflects well-established Western culture. So many diverse and international candidates may not score well on an EQ assessment, even though they could clearly do this particular job. Next, consistency and reliability are major issues. Because many of the elements of EQ cannot be clearly defined and because many of them are internal (i.e., self-awareness, empathy, and motivation). They are not easy to accurately measure. The reliability of the tool suffers even more because EQ assessment is often done during the interview, and most interviewers are not trained in how to measure EQ. Reliability also suffers whenever all interviewers fail to agree in advance on which EQ factors are the most essential and what level of each factor will be necessary to do this particular job at this organization. And finally, the accuracy of the measurement can suffer when candidates purposely “fake” their answers to appear more emotionally intelligent than they actually are. So because of its discrimination, reliability, validity, and measurement issues, it’s extremely hard to justify using this assessment tool during hiring.

Tool #5 – Brainteaser/puzzle questions… don’t predict, but they do confuse – brainfteaser questions are mental puzzles that are verbally given to a candidate during their interview process. And in almost all cases, these mental puzzles are purposely not connected to the job. It is this last fact that frustrates and raises the anxiety level of most interviewees. For years, these brainteaser-type questions were used by and even championed by Microsoft and Google. Fortunately, Google research found that these brainteaser questions are “a complete waste of time.” You will also likely find that the tool has a serious reliability problem because everyone does not agree on the correct answer before the question is asked. And that, unfortunately, means that the same answers, provided by the same candidate, to the same puzzle will almost always get completely different scores from each of the assessors of the question. Finally, the use of these questions may be discriminatory against diverse candidates who might not have a strong math education. So, among all of the 5 “worst” assessment tools, this one has the most data that supports completely avoiding it. 

—————————————–

How Using A “Worst” Assessment Tool… Will Cause You To Lose Top Candidates

Using any of these “worst” assessment tools may cause you to lose as many as 40% of your top candidates. Below, you will find the four different ways in which you will lose top candidates. 

• You will lose many top candidates when they… drop out after encountering a “worst” assessment tool – top candidates are smart. So, when they learn you are using one or more silly or “worst” assessment tools. Many will quickly judge that you are not a well-managed company. So, they will prematurely drop out of your hiring process before the end of their interviews. And, of course, dropping many top candidates will reduce the odds that you actually end up hiring a top candidate.
• The tool may… unnecessarily screen out top candidates – any “worst” assessment tool can, at least in theory, identify faults or weaknesses in a top candidate. And when a candidate receives an unacceptably low assessment score on a “worst” assessment tool. The low score may cause your screening process to reject this otherwise excellent candidate automatically. When recruiting proactively screens the candidate out, you will forever lose your chance of hiring this top candidate who was only found to be disqualified by a tool that should never have been used in the first place. 
• Mixing good/bad assessment results will confuse managers, and that may lead to new-hire-failure – when hiring managers are faced with a mix of assessment results. Some came from accurate assessment tools, while others came from inaccurate “worst” assessment tools. And if the manager making the hiring decision mistakenly treats the results from both categories as equal. You increase the odds that a confused hiring manager will mistakenly pass on a top candidate and instead choose one of the weaker candidates. So this mixing of not non-equivalent results might mean you will end up with a failed new hire.
• You will lose future top candidates… because they will be discouraged from applying – when top candidates who were initially interested in your company hear on social media and glassdoor.com that you still tolerate using outdated and invalid assessment tools. Many of these likely candidates will decide not to apply. And unfortunately, they will also tell their friends to do the same thing. This will, of course, reduce the percentage of applicants that will be highly qualified. And that lower percentage will, unfortunately, reduce your chances of eventually hiring a top-performing candidate.

If you only do one thing – check the hiring records of a few recent hires who have been performing well in their new job. Next, check to see if any of the “worst” five assessment approaches were used on them. And in the cases where they were used. See if any of the scores on these worse tests accurately predicted their high performance on the job. However, don’t be surprised when you find that these “wrong” assessment tools actually contributed nothing to an accurate hiring decision.

Final Thoughts

Like it or not, we are all part of a recruiting profession that now has to adapt to a seemingly endless chain of sudden problems, both in the business world and in technology. As recruiting becomes more data-driven, all smart talent managers will need to shift away from relying 100% on academic studies to prove “what actually works,” not generally but in their own organization. That means that recruiting departments will need to begin conducting their own internal studies to show which assessment tools do/do not predict whether a candidate will be a top-performing new hire. And until they have that specific data for their own organization, no one should be allowed to use any of these “worst” assessment tools. 

Next, because of these rapid changes, today, no one in TA should be allowed to assume for even a minute that a candidate assessment tool that worked well in the 20^th century will continue to be effective in the 21^st century. Instead, in my view, what is needed is a healthy, even cynical, expectation that every recruiting tool will automatically become less effective every quarter. And as a result, no one in TA should ever again be allowed to utter a single change prevention excuse. Including “but we have been using it for decades” or “but the managers like it.” Finally, that means that beginning today every assessment tool must be assigned a “use by date.” After that date, the tool will have to be updated, scrapped, or replaced by technology. Any questions?

Author’s Note

• Please share these ideas by sending this article to your team/network or by posting it on your favorite media. 
• Next, if you don’t already subscribe to Dr. Sullivan’s Aggressive Talent Management article, you can do that here.
• Also, join the well over 20,000+ that follow or have connected with Dr. Sullivan’s community on LinkedIn.