Guide: Gather data and measure decisions

Measure the impact of interventions

Google has designed several interventions, redesigned processes, and developed education programs throughout the company's (continuing) unbiasing journey. It's been essential to measure the impact of these initiatives to ensure that they lead to beneficial change for Google and Googlers. Researchers working on diversity and inclusion have found that, unfortunately, the positive intent of many interventions does not necessarily bring about positive outcomes, and that many efforts have unintentional negative effects. It’s therefore important to roll out changes in a systematic way, and be diligent when measuring impact.

Rolling out programs is best done in an experimental fashion, when possible. Treat HR interventions like a medical researcher treats a drug trial: have a treatment group and an equivalent control group, hypotheses, a data collection period, an analysis comparing groups, and quantifiable outcomes. Google ran an experiment to test the impact of Unconscious Bias @ Work and reached well-founded conclusions thanks to the systematic roll-out of the program. Sometimes an A/B trial is not possible, and in that case it’s advisable to consider longitudinal research. In other words, track changes in your target population over time.

Collecting the right data to test program effectiveness is important, but surprisingly difficult. Every intervention needs to have well-articulated, specific objectives, and measurement needs to be directly tied to these objectives. Inexact measures can lead to ambiguous results. It is important to be deliberate in defining success and how to measure outcomes.

Is this guide useful?

mood mood_bad