Dr John Sullivan Talent Management Thought Leadership

Newsflash: Powerful new data shows that interviews are horrible at identifying the best candidate! Where horrible is defined by interviews failing to select the best candidate over 90% of the time. However, you can dramatically lower this “ugly failure rate.” If you use data to restructure your interviews and then almost overnight, you can expect to see a measurable improvement in your overall recruiting results.

Realize that the basic premise is illogical

Imagine you’re a seasoned chef tasked with judging a cooking competition. But instead of tasting the dishes, you’re only allowed to interview the contestants about their recipes. Sounds absurd, right? Yet this job interviewing process is precisely how we’ve approached the lion’s share of hiring for the past 200 years. (Psychology Today).

You Probably Don’t Know It, But Interviews Are Silently Killing Your Hiring Results

I call interviews “silent killers” because even though everyone uses them and most consider them harmless (they are not), I have found that interview users will only pay more attention to them when they realize that…

“Of all major candidate assessment processes, interviews are by far the most damaging.”

Yes, interviews are incredibly damaging. They are significantly more damaging than the second and third weakest assessments (accepting the multiple lies/exaggerations in resumes and the inaccuracy of most standard reference checks). So, in my view, recruiters and hiring managers need to take this newly released data as an alert and a warning that it’s time to redesign your interviews and begin relying on data to determine all future interview content. 

A Snapshot View Of Data-Driven Interviews

After a data-driven redesign is completed, interviews will still be used. But there will be four significant changes.

• Most traditional interview questions (i.e., tell me about your strengths) will be dropped.
• The remaining interview questions will focus on asking candidates, “How would you solve this current job-related problem?” 
• Much of a candidate’s skill and capability assessment will be shifted to validated outside of the interview assessment tools and methods.
• Finally, much more of the total interview time will be devoted to convincing and selling the candidate on the job and the company. 

The Compelling Business Case For Switching To Data-Driven Interviews

Obviously, it doesn’t make sense to change this established assessment practice without ample data and evidence showing that interviews fail miserably in accurately identifying the best candidate for the job. So, for those who currently embrace interviews without a second thought, here are the 3 most compelling reasons why you should consider shifting to what I call “data-driven interviews.” 

• Startling new data reveals the painfully low likelihood that an interview will identify the best candidate – in research just published in Psychology Today. Timothy Wingate revealed that “91% of the factors that determine how a candidate will perform on the job are completely missed by the interview process.” This means that “interviews only explain about 9% of the variance in predicting future job performance.” For those unfamiliar with variance, in layperson’s terms, it would be accurate to say that out of 100 job openings. Interviews would identify the best candidate less than 10% of the time, meaning they would fail to accurately identify the best candidate a whopping 90% of the time. This would also mean that Google research was correct when it determined that standard interviews don’t select the best candidate better than a coin flip. 

Note: You can access Wingate’s interview effectiveness research article here. 

• Because interviews are so heavily weighted, they are a primary contributor to weak hiring results – interviews are a primary contributor to weak hiring. First, because interviews are used so often. But also because they carry such a heavy weight in the candidate selection decision (interviews are often the sole determinant of the finalist). So, it’s not surprising that I have found that faulty interviews are a primary cause of why a whopping 46% of new hires fail. They are also a primary contributor to the fact that “Companies fail to choose the candidate with the right talent for the job 82% of the time”. 

• You must fully understand the finality of interview scores – many assume that interviews are harmless. However, they are not. Interviews serve as “an absolute gate” (an elimination factor), which means that even though a candidate was determined to be qualified based on their resume. You will never know if they don’t score high on a single interview (often because of anxiety or an inaccurate interview process). Because in the absence of any external skills test scores to counter their low interview scores. The candidate will be permanently eliminated from consideration. And there won’t be an alternative way to find out if the candidate’s only real weakness is that they are poor interviewers. Because of this finality, your interviews must be extremely accurate and data-driven.

The Recommended Actions For Developing Data-Driven Interviews

If you decide that improving the accuracy of your organization’s candidate assessment is essential, here are my top six recommended action steps for developing an accurate data-driven interview process.

Recommended Action #1 – Dramatically reduce your use of standard interview assessments – although you should continue conducting interviews. You must reduce the interview portion where you make subjective candidate assessments based on their answers to traditional interview questions. This means eliminating most standard interview questions that don’t work because they can be practiced in advance. Unless you are a trained psychologist, it also means that you must stop trying to assess the candidate during interviews in areas that simply can’t be accurately assessed (i.e., attitude, energy, commitment, honesty, and intelligence). That also means that interviewers must stop trying to assess a candidate’s facial expressions, body language, handshake, eye contact, dress, and tone of voice. 

Recommended Action #2 – Shift from behavioral interview questions to “How will you act today” questions – even though behavioral interview questions are quite popular. You must realize that they are flawed in several ways. First, behavioral questions are historical (i.e., they cover how the candidate acted in the past). And in our rapidly changing business world. The way a candidate acted in the past may no longer be appropriate (and may even be damaging). Next, behavioral interview answers cover how the individual acted at another time and in a completely different company. Instead, you need to know how the candidate will act today, in this job, and in our company’s culture. Finally, of course, their answers to behavioral questions are likely to be self-serving, and many will take credit for things they didn’t actually do. So, you need to minimize the use of behavioral interview questions during your interviews. And instead, begin using questions that reveal how this candidate would act today. I call these types of questions “How will you act today” questions. These questions ask the candidate to “walk us through the steps” on how they will apply their skills, handle job responsibilities, and handle situations. That is likely to occur in their new job. Some examples of “How will you act today” interview questions include:

• Walk us through the steps on how you will handle conflicts with coworkers related to quality work.
• Walk us through the steps on how you would measure the quality of your work on this job.
• Walk us through the steps on how you would prepare for and assume a leadership role (when asked to by your manager).
• Walk us through the steps of your action plan for your first month on this job.

Recommended Action #3 – Also begin using “How will you solve this problem” interview scenarios – the last section covered how the candidate will handle job-related tasks, duties, and situations. However, the ability to solve problems is important in today’s project environment. You should also begin providing interviewees (and especially candidates for professional jobs). With “How will you solve this problem” scenarios.” The goal of these scenarios is to determine how the candidate goes about solving a problem. These problem-solving questions start with the interviewer briefly outlining a real problem the new hire will likely face during their first month on the job. Then, the interviewer asks the candidate to “walk us through the major steps” they would take to find the best solution to this problem (an actual solution isn’t expected). During and after the candidate’s answer, interviewers can ask follow-up questions. Each interviewer is then asked to judge the candidate based on the steps that they included, the order of those steps, and any critical omissions. Below are some sample “solve this problem” scenarios.

• Building relationships quickly is critical, so can you walk us through the steps that you will take to build relationships during your first month?
• Because prioritizing problems is critical, walk us through the steps that you would take in order to prioritize your major problems. 
• Because we need to add new technology continuously, walk us through the steps of the process that you would use to determine which new technology to purchase.
• Because we need employees who continuously learn, please outline the elements of your action plan for continuously learning in this job.
• You will encounter customer conflicts in this job, so walk us through the major elements of the conflict resolution template that you will use.
• Provide us with a big-picture forecast of the future of this job and our industry to show us that you practice forward thinking.

Note: You can find more details and examples of “solve this problem” interview scenarios here.

Recommended Action #4 – Increase your use of validated outside of the interview assessments – because there are a multitude of assessment areas that cannot be accurately measured during an interview. A major component of the data-driven interview model is the increased use of validated assessment tools applied outside the interview. The assessment approaches I found most accurate and job-related include technical skills tests, whiteboard exercises, hands-on exercises, tryouts/auditions, internships, VR job simulations, peer interviews, and short opportunities to work directly with the team. At the same time, be aware that many popular assessment tools shouldn’t ever be used. Those include selecting the candidate that “you would most like to have a beer with.” In addition to assessing emotional intelligence, brain teaser questions and personality tests are used. You must also be extremely careful whenever you try to assess “the fit” of a candidate. Finally, remember that the primary factor that makes data-driven interviews so effective is the fact that no assessment question or approach can even be considered without seeing and analyzing the reliability and validity numbers of the approach. 

Recommended Action #5 – Allocate significantly more interview time to exciting and selling the candidate – interviews have two primary goals. The first is to assess the candidate (which interviews are not good at). However, the second primary goal of interviews is to convince and sell the candidate. Without effective selling and building trust, many qualified candidates will either prematurely drop out of your interview process or won’t accept your job when offered. So, failing to sell the candidate in a tight job market is a primary drag on recruiting results. And that coupled with the fact that interviews are actually the most effective platform for selling a candidate. You should devote as much as 50% of the allotted interview time to identifying this candidate’s needs (i.e., their job acceptance criteria) and convincing them that this new job will meet those needs. Finally, sales training is the “secret sauce” for improving candidate selling. Training becomes essential because, in reality, most recruiters and only a small percentage of hiring managers actually excel at selling. So, you are the first step in improving the selling component, and you should be educating hiring managers on its importance and how to do it. However, you should also require every recruiter to complete a candidate selling course at least every two years.

Recommended Action #6 – Begin tracking your hiring failures and successes – even with a hiring failure rate that often exceeds 50%. Few large corporations have a formal process for identifying and assessing their hiring failures and successes. Without knowing what assessment tools accurately predicted your top-performing new hires and your hiring failures. Only praying will allow you to improve your hiring results. So, it is essential that you clearly define and track both your hiring successes and your failures. Incidentally, forget all the scary but silly arguments you keep hearing about the difficulty of measuring the quality of hire. Because measuring output quality literally occurs every day in every business function (except HR). So, to avoid arguments, I recommend you begin using the phrase “identifying top-performing new hires” and don’t even mention quality of hire.

If Interviews Are Such Bad Predictors, Why Does Everyone Continue To Use Them?

I have noted for years that “recruiting changes at the speed of rock.” So it’s not surprising that few in recruiting leadership are even seeking, no less adopting, any significant change in their interview process. If you’re curious, here are some reasons why interviews remain unchallenged.

• Interviews don’t change because most are unaware of the common problems with them – I have found over 50 major problems with traditional interviews. Still, most recruiters and managers are unaware of them. You can find my comprehensive list of these top interview problems here.

• Most interviewer users are not data-driven – because most recruiting leaders and recruiters are data-driven. They aren’t likely to see (or even understand) the data that reveals what works and what doesn’t.
• Recruiting is handcuffed by tradition – Unfortunately, I have found that interviews are “the sacred cow of recruiting.” So, you can expect that any proposed changes will meet fierce resistance from managers, but mostly from HR.
• Interviews are favored because they require little preparation – unlike skills tests, whiteboard exercises, and job simulations. They don’t require (or require) much preparation, which makes them extremely popular with managers and recruiters. 

Final Thoughts

In my many decades of recruiting, I have found very few times when a research study has clearly highlighted a serious problem in recruiting. It simply can’t be ignored. Fortunately, this is one of those groundbreaking studies. So, I urge you to consider its results and then modify your candidate assessment approach to continually become more data-driven. After you transform your interviews, you will finally be able to stop hiring candidates who excel at interviewing but who can’t perform on the job!

Note for the reader

This is the latest in Dr. Sullivan’s 25 continuous years of weekly talent articles. Access his thousands of articles here.

Please help to spread his ideas by sharing this with your team/network and by posting it on your favorite media.

And, you can subscribe to his weekly Aggressive Talent Management articles here or by following him on LinkedIn.