Once they understood what constituted a team at Google, the researchers had to determine how to quantitatively measure effectiveness. They looked at lines of code written, bugs fixed, customer satisfaction, and more. But Google’s leaders, who had initially pushed for objective effectiveness measures, realized that every suggested measure could be inherently flawed - more lines of code aren’t necessarily a good thing and more bugs fixed means more bugs were initially created.
Instead, the team decided to use a combination of qualitative assessments and quantitative measures. For qualitative assessments, the researchers captured input from three different perspectives - executives, team leads, and team members. While they all were asked to rate teams on similar scales, when asked to explain their ratings, their answers showed that each was focused on different aspects when assessing team effectiveness.
Executives were most concerned with results (e.g., sales numbers or product launches), but team members said that team culture was the most important measure of team effectiveness. Fittingly, the team lead’s concept of effectiveness spanned both the big picture and the individuals’ concerns saying that ownership, vision, and goals were the most important measures.
So the researchers measured team effectiveness in four different ways:
- Executive evaluation of the team
- Team leader evaluation of the team
- Team member evaluation of the team
- Sales performance against quarterly quota
The qualitative evaluations helped capture a nuanced look at results and culture, but had inherent subjectivity. On the other hand, the quantitative metrics provided concrete team measures, but lacked situational considerations. These four measures in combination, however, allowed researchers to home in on the comprehensive definition of team effectiveness.