Measurement is key for successful unbiasing. Without understanding and tracking your undertakings, it's hard to know where to focus or how your efforts are faring. Here are three measurement categories Google considers when approaching unbiasing:

1. Outcomes of quantifiable decisions that affect employees
2. Beliefs and experiences about the culture, inclusion and the work environment
3. Impact of unbiasing interventions

When possible, it's valuable to record the outcomes of formal people decisions - in a database, over time, with accessibility to the appropriate analysts. Here are some metrics Google started to collect:

• Hiring (e.g., applicant, feedback, interview score, offer data)
• Performance (e.g., ratings)
• Promotion (e.g., nominations, decisions)
• Pay (e.g., base pay, merit increases, bonus)

The Google People Analytics team analyzes aggregate trends in each of these data sources by gender and, if the sample size is large enough, by ethnicity. It's important to incorporate as many relevant control variables as possible to ensure results of these analyses can be interpreted. The team works to compare people in the same jobs, similar levels, and account for the factors that could affect each outcome variable.

By monitoring these people decisions regularly, the team was able to identify discrepancies in the promotion process. Google monitors and reports promotion rates every cycle, and splits promotion rates by gender to check for differences. In 2010, the team found a difference between men and women software engineers. In Engineering, Googlers can self-nominate for promotion when they feel ready to move to the next job level. In one cycle, Google's data showed that junior, female software engineers were not getting promoted at the same rate as their male counterparts. When digging into this, the People Analytics team found that the problem stemmed from differing self-nomination rates. Men, who in many cultures are typically more comfortable self-promoting, were nominating themselves at higher rates than their equally qualified but, on average, less self-promoting female peers. To solve this, a respected senior leader shared the data with Googlers encouraging all engineers to self-nominate if they were ready, and told managers to keep their eyes open for promo-ready Googlers. Following the nudge, promotion rates equaled out.

It’s not only important to understand what decision was made, but also how that decision and surrounding interactions affect employees. The Google team is working to understand how unconscious bias affects people’s beliefs, experiences and attitudes at work, and promote a climate of inclusion and a sense of fairness.

Cultural assessments are very important, as they help measure whether employees feel valued, included and treated fairly, and whether they feel the organization as a whole is a place where they can thrive. These are important factors to assess. One cannot presume that a lack of gender or ethnicity differences in the outcomes described above (for example) are suitable proxies. To assess experiences and beliefs, one must ask the right questions. At Google, the primary vehicle for doing so is the annual employee opinion survey, Googlegeist. Survey questions may touch on the following:

• Inclusion: Do female and male, under-represented minorities (URM) and non-URM, Googlers in each office location, feel equally included and valued? Comparing responses to questions like - “I feel comfortable being myself at work, even when I am different from others,” and, “Google is a place where all types of Googlers (e.g., all genders, ethnicities, cultural backgrounds) can succeed to their full abilities” - allow analysts to assess those feelings of inclusion. Google also focuses on the relationships employees have with their managers. Survey questions such as - “The actions of my manager show that he/she values the perspective I bring to the team, even if it is different from his/her own,” and “My work group has a climate in which diverse perspectives are valued” - help provide insight as well.
• Fairness: Employees' perceptions of people-decisions are as important as the actual outcomes of those decisions. To assess those perceptions, Google asks employees to share feedback on whether or not they believe the promotion process is fair, whether people are properly recognized for their contributions, and whether or not compensation is fair.

If you’re not collecting the data, someone else might be.

Google has long enjoyed celebrating birthdays of famous scientists, explorers, and innovators on the company homepage. Playing around with the logo and creating Google Doodles has been a fun way to celebrate the achievements of important historical figures.

But, as it turns out, Google was mostly celebrating the achievements of men. This was pointed out by a STEM educator who tracked all the Doodles and posted a gender breakdown and an open letter imploring Google to improve.

The Google Doodler team, which is about half women, was shocked when they saw the data and immediately went about setting up processes to track and improve the gender representation in their work. One year later, after being exposed to the harsh truth, the gender representation in the 2014 doodles was 50%. The Doodle team continues to track Doodle diversity today.

Google has designed several interventions, redesigned processes, and developed education programs throughout the company's (continuing) unbiasing journey. It's been essential to measure the impact of these initiatives to ensure that they lead to beneficial change for Google and Googlers. Researchers working on diversity and inclusion have found that, unfortunately, the positive intent of many interventions does not necessarily bring about positive outcomes, and that many efforts have unintentional negative effects. It’s therefore important to roll out changes in a systematic way, and be diligent when measuring impact.

Rolling out programs is best done in an experimental fashion, when possible. Treat HR interventions like a medical researcher treats a drug trial: have a treatment group and an equivalent control group, hypotheses, a data collection period, an analysis comparing groups, and quantifiable outcomes. Google ran an experiment to test the impact of Unconscious Bias @ Work and reached well-founded conclusions thanks to the systematic roll-out of the program. Sometimes an A/B trial is not possible, and in that case it’s advisable to consider longitudinal research. In other words, track changes in your target population over time.

Collecting the right data to test program effectiveness is important, but surprisingly difficult. Every intervention needs to have well-articulated, specific objectives, and measurement needs to be directly tied to these objectives. Inexact measures can lead to ambiguous results. It is important to be deliberate in defining success and how to measure outcomes.