Use structured interviewing
Introduction
Structured interviewing simply means using the same interviewing methods to
assess candidates applying for the same job. Research shows that structured
interviews can be predictive of candidate performance, even for jobs that are
themselves unstructured. Google uses structured interviewing — using the same
interview questions, grading candidate responses on the same scale, and making
hiring decisions based on consistent, predetermined qualifications.
So why don’t more organizations use structured interview questions? Well, they are hard to develop. You have to write them, test them, and make sure interviewers stick to them. And then you have to continuously refresh them so candidates don’t compare notes and come prepared with all the answers. Research has also shown that structured interviews aren’t more frequently used because, in general, interviewers everywhere think they’re good at interviewing and don’t need the help. Surely many of us like to think we’re excellent judges of character.
But when it comes to hiring, don't trust your gut. Research shows that during first encounters we make snap, unconscious judgments heavily influenced by our existing unconscious biases and beliefs. For example, in an interview context, without realizing it, we shift from assessing the complexities of a candidate’s competencies to hunting for evidence that confirms our initial impression. Psychologists call this confirmation bias.
Structured interviews can work for organizations of any size, from a small team up to the federal government. The US Office of Personnel Management encourages government agencies to use structured interviews in hiring and offers a set of free resources for anyone to use.
So why don’t more organizations use structured interview questions? Well, they are hard to develop. You have to write them, test them, and make sure interviewers stick to them. And then you have to continuously refresh them so candidates don’t compare notes and come prepared with all the answers. Research has also shown that structured interviews aren’t more frequently used because, in general, interviewers everywhere think they’re good at interviewing and don’t need the help. Surely many of us like to think we’re excellent judges of character.
But when it comes to hiring, don't trust your gut. Research shows that during first encounters we make snap, unconscious judgments heavily influenced by our existing unconscious biases and beliefs. For example, in an interview context, without realizing it, we shift from assessing the complexities of a candidate’s competencies to hunting for evidence that confirms our initial impression. Psychologists call this confirmation bias.
Structured interviews can work for organizations of any size, from a small team up to the federal government. The US Office of Personnel Management encourages government agencies to use structured interviews in hiring and offers a set of free resources for anyone to use.
Know the components
Structured interviewing is a process where candidates are asked a consistent
set of questions and clear criteria are used to assess the quality of
responses. Structured interviews are used all the time in survey research. The
idea is that any variation in candidate assessment is a result of the
candidate’s performance, not because an interviewer has higher or lower
standards, or asks easier or harder questions.
Google's hiring team has also found that structured interviews cause both candidates and interviewers to have a better experience and are perceived to be more fair based on feedback surveys and user studies.
Google’s approach to structured interviewing consists of four aspects:
1. Using vetted, high-quality questions that are relevant to the role (no brainteasers!)
2. Recording comprehensive feedback of candidate answers so evaluators can easily review responses
3. Scoring with standardized rubrics so that all reviewers have a shared understanding of what a good, mediocre, and poor response looks like
4. Providing interviewer training and calibration so that interviewers are confident and consistent in their assessments
Google's hiring team has also found that structured interviews cause both candidates and interviewers to have a better experience and are perceived to be more fair based on feedback surveys and user studies.
Google’s approach to structured interviewing consists of four aspects:
1. Using vetted, high-quality questions that are relevant to the role (no brainteasers!)
2. Recording comprehensive feedback of candidate answers so evaluators can easily review responses
3. Scoring with standardized rubrics so that all reviewers have a shared understanding of what a good, mediocre, and poor response looks like
4. Providing interviewer training and calibration so that interviewers are confident and consistent in their assessments
Learn the external research
The benefits of structured interviewing, including increased predictive
validity and reduced adverse impact, have been well documented by academic
research over the last 20 years.
In a structured interview, well-trained interviewers ask a set of planned, rigorous, and relevant interview questions and use a scoring guide to make sure their interview ratings are accurate.
Dr. Melissa Harrell, a hiring effectiveness expert on the Google People Analytics team, notes:
"Structured interviews are one of the best tools we have to identify the strongest job candidates (i.e., predictive validity). Not only that, they avoid the pitfalls of some of the other common methods."
Structured interviews tend to be more fair and candidates like them better than personality assessments.
- Structured interviews result in increased predictive validity and decreased differences between demographic groups.
- Structured assessment tools (what the researchers call “behaviorally-anchored rating scales,” and what Google calls “rubrics”) are more predictive than unstructured interviews.
In a structured interview, well-trained interviewers ask a set of planned, rigorous, and relevant interview questions and use a scoring guide to make sure their interview ratings are accurate.
Dr. Melissa Harrell, a hiring effectiveness expert on the Google People Analytics team, notes:
"Structured interviews are one of the best tools we have to identify the strongest job candidates (i.e., predictive validity). Not only that, they avoid the pitfalls of some of the other common methods."
Structured interviews tend to be more fair and candidates like them better than personality assessments.
Read Google’s internal research
Given the strong academic research on the topic, Google's hiring team decided
to experiment with a structured interviewing approach starting with select
groups. The outcomes have been very positive and the team is now expanding the
approach. The team is creating structured interview questions and rubrics for
a variety of roles and training interviewers to use a structured
approach.
What results is the team seeing so far?
What results is the team seeing so far?
- Structured interviews are better at indicating who will do well on the job: Results show that structured interviews are more predictive of job performance than unstructured interviews when comparing interview scores to the performance scores of those hires across functions and levels.
- Interviewers are happier and saving time: Using pre-made, high-quality questions, guides, and rubrics saves on average 40 minutes per interview. Googlers conducting structured interviews reported that they felt more prepared when interviewing the candidate.
- Structured interviews make candidates happier: The team has seen an uptick in candidate satisfaction in feedback scores for structured interview candidates. Interestingly, scores indicated an especially big difference in candidate satisfaction rates when comparing rejected candidates. Rejected candidates who had a structured interview were 35% happier than those who did not have a structured interview.
Define hiring attributes
Before writing any interview questions, it’s good to have a firm
understanding among your team and hiring managers about what you’re looking
for from candidates. A
job analysis
is a useful way to identify the tasks of the role and the attributes and
behaviors that make someone successful in that job.
There are four general attributes that Google looks for when hiring, but each organization can determine the right attributes or competencies for their organization and specific roles.
Here are Google's four attributes:
There are four general attributes that Google looks for when hiring, but each organization can determine the right attributes or competencies for their organization and specific roles.
Here are Google's four attributes:
- General cognitive ability. Google wants smart people who can learn and adapt to new situations. This is about understanding how candidates solve hard problems in real life and how they learn, not about GPAs or SAT scores.
- Leadership. Google looks for a particular type of leadership called “emergent leadership.” This is a form of leadership that ignores formal designations. At Google, different team members will need to step into leadership roles, contribute, and — just as importantly — step back once the need for their specific skills has passed.
- Googleyness. Google wants to make sure the candidate could thrive at Google, and looks for signs of comfort with ambiguity, bias to action, and a collaborative nature.
- Role-related knowledge. Google wants to make sure that the candidate has the experience, background, and skills that will set them up for success.
Draft your interview questions
Effective interview questions help with assessing candidates. Google's
interview questions often contain an initial prompt, along with several
follow-ups designed to understand the candidate's thought processes. The goal
is to make the questions complex enough that candidates can't solve it by
drawing on job experience alone.
Parts of an interview question:
Interviewers may not be looking to assess a correct answer, but instead want to judge the analytical thought process used to explore, define, and offer a solution to the problem.
Parts of an interview question:
- The initial prompt introduces the scenario. This sentence is clear, concise, and phrased in a way that encourages candidates to explain their process and offer a solution. The scenario is usually a situation that could be realistically encountered on the job but does not require technical or job-specific skills to respond.
- Follow-up questions are predetermined extensions of the initial prompt. They also help elicit a high level of detail from the candidate by encouraging them to thoroughly describe and explain their approach to solving the problem. Refer to your required attributes list to help you identify the ones that the initial prompt does not cover and then design your follow-up questions to address those areas.
Interviewers may not be looking to assess a correct answer, but instead want to judge the analytical thought process used to explore, define, and offer a solution to the problem.
Understand behavioral vs. hypothetical questions
There are two kinds of structured interview questions: behavioral and
hypothetical. Behavioral questions ask candidates to describe prior
achievements and match those to what is required in the current job (i.e.,
“Tell me about a time…?”). Hypothetical questions present a job-related
hypothetical situation (i.e., “What would you do if…?”). This is what some of
these interview questions can sound like:
Use behavioral questions to test how a candidate responded to a past situation and hypothetical questions to assess a response to a future situation. Behavioral questions are good at revealing patterns of behavior and hypothetical questions allow you to see how a candidate would respond to novel situations.
- Behavioral: Tell me about a time your behavior had a positive impact on your team. (Follow-ups: What was your primary goal and why? How did your teammates respond? Moving forward, what’s your plan?)
- Hypothetical: Imagine you're working on an email product and a competitor starts charging a $5 monthly fee for their product. How would you assess the situation and what recommendation would you make to your team? (Follow-ups: What factors would you take into consideration when making your recommendation? What are the pros and the cons of your recommendation? How would you assess if this was a sustainable model moving forward? What impact would this have on the organization as a whole?).
Use behavioral questions to test how a candidate responded to a past situation and hypothetical questions to assess a response to a future situation. Behavioral questions are good at revealing patterns of behavior and hypothetical questions allow you to see how a candidate would respond to novel situations.
Use a grading rubric
For a structured interview question, creating a grading rubric can help assess multiple candidates' answers and to fairly and consistently compare applicant responses.
For the attribute or quality the question is designed to test, Google often documents with illustrative examples what a poor, mixed, good, and excellent answer would cover.
Have interviewers take detailed notes of the responses. This allows the independent hiring committee to review and verify the assessments of the interviewers.