“Interviews are arguably the most difficult technique to get right, because interviewers should stick to questions that predict good hires — mainly about past behavior or performance that’s relevant to the tasks of the job — and ask them consistently across candidates. Just winging it and asking whatever comes to mind is next to useless.”
— Peter Cappelli, “Your Approach to Hiring Is All Wrong”
Interviews are our primary way to assess the quality of new hires. However, according to talent expert Dr. Brad Smart and others, the evidence suggests that in the majority of hiring situations, interviews are woefully inaccurate in predicting the high performers that companies had expected and hoped for. In the end, those companies are disappointed with more than 80 percent of the people their hiring and interview methods produced!
There are at least three factors that come into play to determine if your hiring and interview process is effective at all:
- The quality of your interviewing methodology and its ability to consistently predict good hires
- The consistency of interviewer quality and the questions asked across different interviews and candidates
- The presence or absence of measurement; most firms don’t track and hold managers accountable, and it’s impossible to tell if your hiring and interview process is effective if you can’t determine what percentage of the candidates you select become good hires.
At most companies, the interview process does not have much structure or uniformity, and very little training or certification is given to produce consistently good interviewers. In some cases, different interview methods are used in different departments or business areas, often at the whim of the hiring manager.
Let’s look at these different interview methods, ranked from least accurate to most accurate:
1. Unstructured Situational Interview
The unstructured situational interview is quite common for four reasons. One, it doesn’t require much preparation because the interviewer can take the interview in any direction they want. As such, it is great for undisciplined organizations.
Two, it appeals to hiring managers’ vanity that they are great judges of people. (Research shows that they are not!)
Three, it appeals to a manager’s desire to have the freedom to wing it.
Four, asking avant-garde questions feels clever and fun to managers, and they get satisfaction from candidates who confirm their own biases with their responses. (“Oh my goodness, I feel the same way!”)
The unstructured situational interview is wrought with problems. Due to its unstructured nature, interviewers decide what questions to ask on the fly. This causes inconsistency from the get-go in each and every interview.
Inevitably, since the interview is unstructured, hypothetical questions about hypothetical situations yield hypothetical answers. In other words, hypothetical in equals hypothetical out. These inane questions are useless classics, such as “Tell me about yourself” and “Why should we hire you?” When a concrete situation is added to these questions, it is itself imaginary: “How would you handle a difficult coworker?” The answer to this question is based only on theory, not on fact.
Ultimately, the unstructured situational interview is of little value because it asks candidates random questions to get a feel as to whether they fit the organization. This is no more than a gut feeling, which is extremely inaccurate in predicting successful hires. In fact, studies show that unstructured interviews have low validity correlation, with a r2 (coefficient of determination) of only about .14 to .31, meaning they can predict only about 14-31 percent of a candidate’s success factors.
In summary, the unstructured interview is of little use. It is candidly used because of its expediency and manager hubris. I would recommend that this method be used only as part of a sequential multistep interview process. It can be the initial meet-and-greet-get-to-know-you interview that is followed up with a proper structured interview later in the process.
2. Case/Puzzle Interview
The case or puzzle interview is an interview with one or more hypothetical brainteaser-type questions. This format was popularized by consulting firms like McKinsey and Company, Boston Consulting Group, and Bain and Company with the intent to uncover a candidate’s thought process. At best, these interviews are presented to candidates in the form of a detailed case study that contains a series of sequential questions about a theoretical business situation. At worst, they are a set of frivolous questions like “What would be the best way to sell 10,000 pencils?” or “Our client wants to expand in a new market segment. What factors should they consider?” or the infamous and totally ridiculous “How many ping-pong balls will fit inside a 747?”
The theory behind case interviews is that they offer a way to test a candidate’s analytical skills within a business context. The fundamental issue is that since both the context and the candidate’s answers are hypothetical, they show little correlation to what the candidate has actually achieved, which is the best predictor of success. The answers say nothing at all about a candidate’s underlying character.
Furthermore, a smart but unscrupulous individual can easily figure out how to do well in these interviews. In fact, most of the big consulting firms have sample case interviews on their websites so you can practice! In other words, a complete narcissist who is analytically inclined can spout out some impressive theories, but there is nothing in the interview that would indicate this person would actually be a desired team member for your organization.
A renewed interest in case and puzzle interviews and their ilk, including fancy game-based assessments, seems to be arising in the quest for a smart-sounding talent assessment shortcut that fits high-tech companies’ perceived “hip knowledge worker” persona. After all, it feels chic to use brainy techniques to select smart people, particularly in a company with a high regard for the intellect of its knowledge workers. The automated versions of these techniques also sell the promise of gaining deep insights quickly and at scale.
However, the case or puzzle interview and its automated offspring haven’t shown any recognized correlations that indicate documented success in selecting successful hires. Instead, these interviews appear to be more a case of trying something cool and trendy to avoid the arduous nature of using a proven interview technique that is actually validated.
Case and puzzle techniques are grounded in the hypothetical, and therefore, they have no place as the centerpiece of your candidate selection process. At best, they should be used as a tiebreaker to augment a better interview process. Use the case or puzzle interview with caution due to its lack of documented validation.
For more expert recruiting insights, check out the latest issue of Recruiter.com Magazine:
3. Structured Situational Interview
A structured situational interview is perhaps the most commonly employed interview style. That is likely because it has been around for a long time, and most hiring managers who are not formally trained in interview skills experienced this interview style when they were job candidates once upon a time.
In situational interviews, candidates are asked to respond to certain situations they may face on the job, hence the name. Compared to the unstructured situational interview, the structured situational interview adds a modicum of structure that promotes consistency between candidates and provides a better and fairer way to assess each applicant. In other words, the same questions are typically asked of each candidate and in the same manner. Best practice, as in all interviews, is to take detailed and precise notes on what the candidates actually say.
The other distinction is that the structured situational interview is famous for questions that focus on how the candidate would handle a hypothetical situation. Most questions take the format of “What would you do if … ?” Consider the following:
- How would you handle an angry customer?
- How would you sell our product lineup?
- Why do you think you are the ideal candidate for this position?
- What do you think is our No. 1 challenge?
- Let’s say you are asked to choose between two competing priorities. How would you prioritize?
Note that all of these questions are based on theoretical situations and ask the candidate to explain what they would do, as opposed to what they have actually done.
As background, the structured situational interview is typically conducted as a second step, usually after some sort of resume screen and often by a different party. This is a shame because the resume provides a useful context in which to base factual questions grounded in real-world performance.
The theory behind situational interviews is that they encourage the candidate to think about the role for which they are being interviewed while catching lesser candidates off guard in the process. In truth, situational interviews are mired in fantasy and the quixotic pursuit of great questions to ask the candidate.
Interviewing is hard work. Asking detailed behavioral-based questions that uncover a candidate’s work product makes many untrained interviewers uncomfortable, which is why they stop short at the more whimsical situational questions.
4. Work Skills Tests and Cognitive/Psychometric Assessments
While not the Holy Grail and not technically interviews, both work skills tests and cognitive/psychometric assessments actually have mid-level predictive ability with validity correlations of 29 percent and 26 percent respectively. Used correctly, one or both of these techniques may improve hiring accuracy.
Work skills tests attempt to simulate the actual work of the job. They range from writing exercises to math problems, spreadsheet exercises, editing skills, computer coding tests, and sorting an inbox. Some companies take this a step further and ask candidates to perform a specific technical skill, like welding a seam or operating a piece of equipment like a lathe or excavator.
Cognitive assessments aim to measure both intellect and capacity to learn. Psychometric assessments go one step further and are designed to measure a candidate’s suitability for a role based on required personality characteristics, aptitudes, and cognitive abilities. They identify the candidate’s personality and cognitive abilities and match those against large databases of people who have been successful in similar roles.
While work skills tests and cognitive/psychometric assessments should not replace interviews in your hiring process, they can provide very useful incremental data to increase accuracy in selecting successful hires. I have several clients who use both in addition to an extremely structured behavioral interview, and their hiring systems enjoy extremely good predictive outcomes in selecting top performers. When faced with two or more competitive candidates, these assessments may just prove to be very predictive tiebreakers.
5. Structured Behavioral Interview
The structured behavioral interview is the most accurate and predictive class of interview. In 1993, researchers Vivian Shackleton and Neil Anderson found structured interviews to have validity correlations in the 62 percent range. Specific enhancements to the structured behavioral interview, which we’ll discuss in a moment, can take the predictability even higher.
The hallmark of the structured behavioral interview technique is asking factual questions about the candidate’s past behavior within the context of specific job experiences and results. To make the structured behavioral interview most effective, the interviewer should ask their questions within the context of a specific job accomplishment listed on a resume or other interview guide, as opposed to a more generic open example from anywhere in a candidate’s history.
For example, an interviewer might find out there were management disagreements regarding a corporate turnaround with questions like “What was your specific contribution to XYZ Corporation’s $50 million turnaround?” and “You mentioned there were management disagreements regarding the turnaround strategies. What was your position, and how did you resolve your disagreements with other managers? What happened, and what did you learn from the experience?” These are far more effective behavioral questions than “Tell me about a time you resolved a difference with another manager.” The softball question is far too vague, as the candidate will pick their favorite and most often rehearsed response from anywhere in their history. At this point, the interviewer loses some control of the interview to the candidate. This is an issue because the candidate will have selection bias and will want to guide you to their highlights while steering you away from their failures and mistakes.
As mentioned, when the interviewer sticks to deep, specific behavioral questions for each role the candidate has held, the structured behavioral interview is by far the best predictor of successful hires because a candidate’s actual past performance is the best predictor of their future success. In addition, when good questions are asked with this method, you learn a lot about the candidate’s actual character, including their truthfulness, loyalty, and resilience.
You can get even more predictive and accurate in your hiring with the topgrading behavioral interview — essentially, a structured behavioral interview on steroids:
Summary of Interview Types
Here is a summary of the various interview types, ranked from most to least in terms of hiring accuracy and predictability:
Solo vs. Panel Interviews
The question often arises as to which is better: a sequence of solo interviews or one panel interview? Those who support the solo interview format argue that the lead interviewer in a panel interview can influence the questions as well as the other interviewers. In addition, they argue that this format is more intimidating for the candidate.
The advantages of the panel interview format include consistency in the interview process, as it is led by a trained, skilled interviewer who is good at asking factual behavior-based questions and putting the candidate at ease. The other interviewers in the room take detailed notes and ask questions that build on and go deeper than the lead interviewer’s questions. The panel will debrief as a group immediately following the interview. As this team will potentially interview several candidates for the same position, they are in an excellent position to compare the candidates head to head. The panel interview also offers the advantage of teaching interview skills to all involved.
The panel interview provides consistency, as every interviewer on the panel sees exactly the same candidate performance. It is also faster, as it is very hard to pull a group of solo interviewers together for a debrief. Often, that meeting never properly happens. In contrast, at the conclusion of a panel interview, the team immediately deliberates. In the case of a strong candidate, the team may be in a position to make an offer right away. In recruiting, speed conducted with accuracy wins.
Also, I have found that instead of being intimidated, strong candidates love meeting their potential new peers all at once, and they get a much clearer view of the culture they will be joining as a result of the interview interaction. By assembling a relevant panel, it shows the candidate that you are serious about talent and about them.
Creating teams of high performers — A players — is the No. 1 leadership skill. While most companies have tons of metrics, the paradoxical exception is that most companies do not measure the most important thing — talent — well enough!
Organizations should track the percentage of people they hire who turn out to be the high performers they thought they were getting. The best way to improve your hiring success is to track, document, and provide consistent training on a structured behavioral interview methodology.
Rick Crossland is author of the book, “The A Player.” He works with organizations across the country to transform good companies into great companies. More information on this and other ways to improve your own performance and the performance of your organization and culture can be found in the “The A Player.” More resources are available at www.aplayeradvantage.com.