Fighting incivility with machine learning at Alphabet’s Jigsaw

Fighting incivility with machine learning at Alphabet’s Jigsaw
The dream of the internet is to have a place where people can share their ideas respectfully. In reality, individuals can be so loud, rude, and abusive that serious discourse is impossible. But by enhancing community norms with machine learning we can start to scale civility.

CJ Adams is product manager at Jigsaw, an incubator at Alphabet whose goal is to use technology to help people who are facing oppression. Adams studies incivility in online behavior and at the 2016 re:Work event he spoke about how the methods for encouraging online civility are evolving with the help of machine learning.

A number of methods are currently employed to reduce incivility, including platform rules, real names policies, and flagging of inappropriate behavior. But individually, all of these solutions are extremely limited and are far from solving the problem. Adams suggests that a combination of machine learning and community feedback can identify and reduce incivil online behavior in a human-driven but scaleable way.

Adams highlighted the approach of video game maker Riot Games. Riot Games has tools that allow their community, the gamers, to flag offensive and abusive behavior. This flagging is the basis for a data set used to train an automated system on what is and is not acceptable behavior. This system can then automatically start flagging behavior and provide feedback to players who break the community’s rules. When presented with the information that they were breaking game’s social norms, 50% of players changed their behavior. And when shown the specific behavior of theirs that was problematic (usually a chatlog), 70% of players changed their ways. Riot has taken this technology and applied it to their hiring and people management practices internally to try to flag potentially toxic employees too.

By having machine learning systems learn from the community what is and is not permissible, huge online communities can start to nudge their members to be better and more respectful citizens. Google recently open sourced a powerful library for machine intelligence called Tensor Flow in the hopes that developers everywhere can start to use machine learning to solve online incivility and other problems.

Learn more about incivility and toxicity in the workplace on re:Work: