Riot Games: Assessing toxicity in the workplace
Challenge
Riot Games is the maker of one of the world’s most popular computer games - League of Legends. Nearly 30 million players log in every day to compete in League’s battle arenas; 7.5 million players clash concurrently at peak times. The team-based game is a take on capture the flag (or, more aptly, “destroy the enemy base”). Team dynamics, sportsmanship, and camaraderie are key to giving League players a fun experience.
And so if a player is exhibiting toxic behavior in the game, in the form of written chats, it can have a seriously detrimental impact on their teammates’ experience. If a new player encounters toxic behavior in their very first game, they are 320% less likely to come back again. Toxic behaviors are defined by the community when players report bad behavior and flag offensive chats via in-game tools. There are regional nuances, but toxic behaviors include homophobia, racism, sexism, and other forms of hate speech. Riot has built a sophisticated system to flag toxic behavior and reward and reinforce positive behavior in League of Legends.
And just like in a League of Legends game, research has shown that having a toxic teammate in the workplace can bring down the performance and morale of an entire team, having costly effects for an organization. Having long experimented with combatting bad behavior in their game, Riot’s Talent team wondered whether there was a correlation between in-game toxicity and real-world behavior. All Riot employees (called “Rioters”) play League of Legends and every player has a behavioral profile. The Talent team partnered with their game designers to see if in-game toxicity might predict workplace troubles.
Action
The Riot team hypothesized there’d be a correlation between highly toxic in-game play and workplace toxicity; if a Rioter received lots of in-game complaints, the team assumed they'd have more friction with workplace teammates too. This is not to say Riot had a problem with workplace toxicity. Ranking high on Fortune’s 100 Best Places to Work For list, it was quite the opposite. But as a quickly growing company they were concerned about strengthening and scaling their culture.
Riot looked at the preceding 12 months of gameplay of every employee and discovered there was a correlation between in-game and in-Riot toxicity. They determined that 25% of employees who had been let go in the previous year were players with unusually high in-game toxicity. The most common bad behaviors they found were passive aggression (snarky comments) and the use of authoritative language, sometimes using their authority as a Riot employee to intimidate or threaten others.
Riot also found that a player’s toxicity was a fluid thing and not immutable. Like moods, toxicity levels can fluctuate. Riot could measure and predict toxicity trajectories of players over time, and so they set about seeing if they could improve the player behavior of their employees.
Riot identified the 30 most toxic employees (all of whom were more junior Rioters, new to the working world) and classified them into two categories:
- this person needs a stern warning
- this person should leave Riot, because their in-game chat was unusually toxic
Riot’s Talent team then scheduled meetings with all these Rioters to discuss their in-game behavior. The team brought with them in-game chat logs where that Rioter was exhibiting especially bad behavior. While these logs did factor into the exits of a couple employees who had already had serious problems, the response from almost everyone else was overwhelmingly positive. “Pretty much everyone we spoke with was appalled at their own behavior. We actually received some essays from employees vowing to change their ways and become not just more considerate gamers but better people,” said Jay Moldenhauer-Salazar, Riot’s head of Talent.
Results
Riot is a quickly growing company with managers eager to fill seats, but the organization wants to make sure they don’t make any bad hires. The team is now trying to figure out how to use this in-game information as a signal during the hiring process. Riot asks applicants for their in-game handle during the application process so they can review their gameplay and identify any toxic chats and behaviors.
Riot is experimenting with displaying this information in their applicant tracking system using a simple stoplight code - red, yellow, green. For any applicant flagged “red” (the most toxic), there are sample chat logs so a recruiter or hiring manager can see how an applicant conducts themselves in the very product they’re hoping to work on.
While Riot researchers started by delving into the nasty, negative end of the gameplay spectrum, they're now collaborating with researchers at MIT on a collective intelligence study to see what types of personalities compose a great team. They’re hoping to come up with an understanding of the personalities that make the best team composition, and how they can proactively create high performing, positive teams.