OpenAI's ban of Canada school shooter's account raises scrutiny of other online activity

Canada’s Artificial Intelligence Minister Evan Solomon summoned the company officials to Ottawa this week to explain their safety protocols.

OpenAI's admission that it banned the ChatGPT account of mass shooter Jesse Van Rootselaar months before the 18-year-old killed eight people and herself is drawing more scrutiny to her past online activity and raising questions about whether opportunities were missed to prevent one of Canada's worst-ever mass killings.

OpenAI’s decision not to report Van Rootselaar to police prompted Canada’s Artificial Intelligence Minister Evan Solomon to summon company officials to Ottawa this week to explain their safety protocols.

The shooting in the small British Columbia town of Tumbler Ridge is the ‌latest tragedy in which critics have argued interactions with chatbots may have forewarned of or even encouraged violence. Crime experts noted that while greater scrutiny of AI platforms and social media is necessary, police or other authorities may have missed chances to avert the tragedy.

Police had previously removed guns from Van Rootselaar’s home, though they were later returned. Police also said they were aware of her history of mental health issues. Van Rootselaar began the attack by killing her mother and sibling at home, before shooting dead an educator and five students, while two others ​were hospitalized with serious injuries.

The Royal Canadian Mounted police said the investigation is still active and some questions are subject to relevant legislation or court processes.

“This was clearly a household where there were many problems,” said Patrick Watson, a criminology professor at the University of Toronto unconnected to the case. “But we also need far more scrutiny of the companies that are creating these new platforms, which are essentially becoming a new public sphere with very little accountability.”

In a since-deleted Reddit post, Van Rootselaar said ‌she had been diagnosed with numerous mental health issues, including attention deficit hyperactivity disorder, depression, obsessive compulsive disorder, and was on the autism spectrum.

“I went crazy and burnt my house down, my second time trying shrooms, but still have a ‌desire to try alternatives,” Van Rootselaar wrote.

Van Rootselaar also previously created a game using the Roblox Studio app, involving shooting other characters at a mall.

Roblox told Reuters that Van Rootselaar’s account and its content were removed from the Roblox Studio app the day after the Tumbler Ridge massacre, and the game had only seven visits. OpenAI said in a statement it had banned Van Rootselaar’s ChatGPT account last June ‌after identifying "misuses of our models in furtherance of violent activities" and considering whether to refer her to law enforcement.

The company ‌ultimately decided “the account activity did not meet the higher threshold required for referral,” mainly because OpenAI was not able to identify credible ‌or imminent planning. The company said intervening in these situations can be distressing for young people and their families and may also raise privacy ‌concerns.

Missed Opportunity

Tracy Vaillancourt, a professor at the University of Ottawa who specializes in youth mental health and violence prevention, said OpenAI’s failure to refer Van Rootselaar to police was “a missed opportunity,” but acknowledged there were challenges in protecting users’ privacy.

“People using ChatGPT may worry that it’s going to spy on them, but AI is so powerful that there should be a way to improve how technology and we as a society are able to reduce credible threats,” Vaillancourt said.

Cynthia Khoo, a technology and human rights lawyer, warned, "It would be a mistake to start down a path where AI companies might become deputized as a private surveillance wing of law enforcement,” saying that invasions of privacy would disproportionately hit already marginalized groups.

Van Rootselaar was born male but identified as a female and began transitioning six years ago, police said. A 2023 report from the US government showed that more than 95% of mass shooters are male and that transgender people account for about 2%.

British Columbia Premier David Eby said the Tumbler Ridge shooting could have been avoided if OpenAI had warned authorities about Van Rootselaar’s violent online activity and called ​for more transparency from the tech giant.

“It looks like OpenAI had the opportunity to prevent this tragedy, to prevent this horrific loss ​of life, to prevent there from being dead children in ​British Columbia,” he said Monday. OpenAI said in its statement that the shooting was “a devastating tragedy” and that it was doing all it could to support the ongoing investigation.

“We reached out to law enforcement ​immediately after the identity of the shooter was made public, and we are engaged with the (police) to support their ongoing work," the company said.