The hiring process at Google was designed to be meticulous in order to avoid false positives or toxic hires. We thought that the more Google employees who interviewed a single candidate, the better our hiring decision would be — which meant someone applying to Google could be subjected to over a dozen interviews. This took a massive amount of employee time and also could make for a grueling candidate experience. So, were the thousands of extra interviews actually helping us make the best possible hiring decisions?

Our early People Analytics team decided to look at the data to see how helpful and predictive each interview actually was. The team looked at a subset of our interview data over five years to determine the value of an interviewer’s feedback. At Google, each interviewer assigns an interview score to the candidate. So looking at our data, did the mean of two interview scores substantially improve our ability to predict whether the candidate would receive an offer than if we only looked at one interview score? How about the mean from adding a third interviewer’s score? A fourth? And so on.

It turns out there is a diminishing return on interviewer feedback. Based on their statistical analysis, the team found that four interviews were enough to predict whether someone should be hired at Google with 86% confidence. After the fourth interviewer, the increase in accuracy dropped off dramatically — with each additional interview, the accuracy of the mean interview score’s ability to predict a hire/no hire decision increased less than one percent. While this finding is specific to Google, it mirrors findings from the UK’s Behavioural Insights Team, who recommend having at least three reviewers for each candidate.

The Google team wanted to test whether four-person interview panels could make the same hiring decisions that panels made up of more than four interviewers had actually made in the past. To do this, the team first calculated the correlation between a candidate’s average interview performance score using all the interviews and compared it with a recalculated interview performance score from four interviewers randomly selected from each candidate’s slate. Not surprisingly, the two scores were very highly correlated.

Next, they looked at the hiring outcomes under each condition, specifically, comparing the actual hiring outcome that was based on the full slate of (more than four) interviews to a hypothetical hiring outcome based on four randomly selected interviews. The results were almost identical: 94% of the time, the exact same decision would have been made. We recently used 2016 interview data to retest this finding and found that our initial analyses still hold true: panels of four interviewers make the same hiring decision 95% of the time compared to panels made up of more than four interviewers.

The implications of this research were obvious: We could confidently make hiring decisions with as few as four experienced Google employees conducting interviewers. Armed with this data, Google’s staffing team proposed what we now call the “Rule of Four,” which we use for standard hiring with some exceptions (for example, allowing still-in-training interviewers to practice and calibrate by interviewing real candidates). Our Rule of Four reduced the average time-to-hire by about two weeks, has saved employees hundreds of thousands of hours in interviewing time, and has helped reduce the already stressful process for candidates.