Dr John Sullivan Talent Management Thought Leadership
Selecting HR Metrics Is Unfortunately Not a Scientific Process
Most organizations adopt metrics based on those covered by benchmark reports or that can be easily enabled via their technology providers, instead of determining what they need to discover or prove. As a result, many organizations are burdened with data and reports that offer little in the way of guidance helping them improve their effectiveness. One metric often not fully taken advantage of is quality of hire, which I estimate less than 40% of organizations even attempt to use. Even fewer use the quality of hire derivative, quality of those not hired, because it can very quickly demonstrate how poorly a process performs.
Determine Where Your Recruiting Problems Are Occurring
During an advisory conversation with a recruiting leader at a well-known social networking firm experiencing difficulty achieving hire diversity, I asked “at what step or stage is your recruiting process failing?” I wasn’t surprised when he responded “we don’t actually know, we just know that the overall recruiting process is not producing the results we need.” Like many organizations, this organization lacked well-thought-out metrics that enable both performance reporting and process diagnostics.
Recruiting processes fail because either they do not attract enough top-quality candidates up front, or they fail to accurately identify, assess, and sell those attracted on the job at later stages in the process. Most organizations focus heavily on measuring sourcing effectiveness, but ignore the later stages of the process altogether. One benefit of using a “quality of those not hired” metric is that it focuses exclusively on the back end, where I estimate at least 50% of those organizations not meeting their goals have problems. If you doubt that the problem is post-attraction, ask your favorite agency or executive recruiter what percentage of qualified candidates are lost due to slow or ineffective actions on the part of hiring managers and corporate recruiting processes.
One of the purposes of the quality-of-those-not-hired metric is to force organizations with a high percentage of quality hires slipping away to identify where in their process the talent opts out or gets dropped. There are six post application stages where firms lose top candidates, including:
- Resume screening process — the ATS, a recruiter, or a hiring manager mistakenly screens out top applicants.
- Telephone screen — top applicants rank poorly on their phone screens or their screen cannot be completed, so they are dropped from consideration.
- Interview scheduling — they get frustrated over the number of interviews and dropout or they cannot complete them in time because of scheduling conflicts.
- Interview assessment — they voluntarily drop out before the interviews can be completed, or the interview process mistakenly rates them poorly.
- The offer process — either the process fails to include most of the top applicants on the list of finalists, or they reject the offer.
- Reference checking — even though they are high-quality candidates, they somehow fail the reference/background check.
Since the goal of a good metric is to help you identify what is not working, carefully select and implement at least one metric that can point out failures occurring during the latter stages of your recruiting process.
Focus Only On the Top Candidates
A quality-of-those-not-hired metric can become cumbersome if it attempts to categorize the quality of every applicant who doesn’t get hired. In order to save time and money, narrow your focus to the strategic issue of “what happened to the cream of the crop?” Out of 50 applicants for a single job, there might only be three who were so qualified that a hiring manager would actually regret failing to hire them. I call these individuals “regrettable misses,” and it is these folks that the quality-of-those-not-hired metric aims to highlight.
Action Steps for Developing a “Quality-of-Those-Not-Hired” Metric
If you decide to implement a quality-of-those-not-hired metric, there are several action steps to consider, including:
- Setting goals — I recommend that you set goals for the use of this metric that include: accurately identifying the top three to five “regrettable” candidates; determining what percentage of top candidates become finalists for the position, and determining what percentage of new hires came from the top candidate list.
- Select an evaluation range — this metric should focus solely on reporting the progress of “the very top applicants” who senior managers would regret not hiring. To limit the scope of evaluation, preselect what size of candidate slate will be evaluated. For most jobs, three to five top applicants would be a sufficient number to track. You can also use a set percentage of all applicants (i.e. top 10%) to define what you mean by top.
- Determine when to identify top applicants — identify the top applicants early on in the hiring process so that you will have time to address any issues that emerge before a final hiring decision is made. If you are conducting an audit post hire, you need to make sure that the person doing the initial selection isn’t aware of which individuals were finalists and who was hired.
- Select methods for identifying top applicants — the best method for identifying the top applicants is to have multiple evaluators select a finalist slate that is then merged to create the sample that will be monitored. An alternative approach involves using the profile matching capabilities of your ATS to produce a listing of top applicants. A third possible list segments applicants who come from high-value benchmark firms.
- Report the metric in percentages — the best way to report the quality of “those not hired” metric is in percentages. For example: 66% of all finalists came from the top-ranked list, and 47% of the time a top-five-ranked candidate was hired.
- Identify the stage where top talent slips through — for high priority and mission-critical jobs, after the hiring process is complete, identify at what specific stage in the recruiting process did a top applicant opt out or get dropped from consideration. You can then use that information to improve that stage.
- Identify cause for top candidate removal from consideration — if a significant number of top candidates opt out or are dropped from consideration without becoming finalists, follow up and find out why. If your process screened them out prematurely, recruiters and hiring managers must be questioned to identify what knockout criteria is being applied. If the candidate dropped out on their own, they need to be questioned to see if their early withdrawal could’ve been prevented.
- Keep in touch — separate from the process of calculating the metric, the organization should keep in touch with and build a relationship with the high-quality applicants who you regret missing. Building this relationship will help to ensure that they will favorably consider another opportunity with your firm in the future. Develop an alert system so that the star applicants can automatically receive e-mail alerts whenever a relevant job opens up.
Sample “Quality of Those Not Hired” Report
Here is a sample report illustrating what a recruiting leader or hiring manager might see.
Job Family: ASIC Engineer
Report Period: Q3 2010
Hire Volume: 53
| Metric | Job Family | Organization |
|---|---|---|
| Percentage of top candidates in finalist pool | 53% | 52% |
| Party responsible for removal from consideration | ||
| –candidate filtered out by recruiting process | 17% | 48% |
| –candidate opted out of recruiting process | 83% | 52% |
| Percentage of top candidates who rejected offer | 95% | 22% |
| Percentage of hires from top candidate slate | 31% | 42% |
| Process stage contributing largest slate loss | Interview Scheduling |
Recommended Actions
Candidates reported that more often than not interviews would need to be rescheduled because times initially proposed were no longer available upon confirmation. Many candidates reported that it took recruiting coordinators longer than a week to confirm meeting dates and times. Solution:
- Allocate dedicated time slots to recruiting activities that cannot be booked by other activities more than 24 hours in advance.
- Establish service level agreements that call for manager response to scheduling inquiries within four business hours.
Final Thoughts
If you were a competitive fisherman participating in a pro fishing tournament and you repeatedly landed prize-winning fish, you would be justifiably proud. However, if you repeatedly caught prize contenders but lost them prior to tournament completion, wouldn’t you want to know exactly where and why you kept losing them? That is exactly what the “quality-of-those-not-hired” metric tells you. It reports how often you successfully land a great applicant, but fail to convert them to employee. Your organization can’t attain the highest level of new hire on-the-job performance (quality of hire) if your process allows the highest-quality applicants to be missed.
