Lawsuit claims AI on Google’s school Chromebook helped plan teen’s near-suicide

Parents are suing technology giant Google, alleging the AI Gemini chatbot offered on school Chromebooks helped a young teen plan her near-suicide and funeral.

The lawsuit , filed in California Superior Court on Wednesday, claims a 13-year-old girl, dubbed Emma, chatted with Gemini over the course of several months, prompting a “severe psychological crisis.” The complaint alleges Google’s top executives knew Gemini was dangerous and released it anyway.

“As her Gemini chats grew more frequent… Emma became convinced that she was afflicted by a battery of major psychological disorders and that suicide was her only way out,” the EdTech Law Center, representing Emma and her parents, wrote in a press release.

Lawyers said she exchanged “thousands of messages” with Gemini and after three months, asked the chatbot about various ways to commit suicide. The chatbot allegedly told Emma how to hide her mental health and self-harm from people around her and helped her play a funeral.

A former school counsel who received alerts from the school’s third-party monitoring software told Emma’s parents, who logged into her account and intervened.

José Castañeda, a spokesperson for Google, said the company is reviewing the claims and takes the claims “very seriously.”

“We’ve worked extensively with medical professionals and academic experts to build features that help young people find critical support and trusted resources when they need it,” Castañeda said.

“Our systems aim to prevent users from engaging in harmful conversations, and when people’s queries indicate they may be experiencing a crisis, we show hotline information directly in Search to connect them with real-world support,” he added.

According to Google, the company has worked on its crisis detection system for decades, building certain protections for users in crisis. It also has SafeSearch, a filter for explicit content, which is turned on for all education users under 18.

Emma’s lawyers claim she did not seek out Gemini, but “it just showed up in Google search one day.”

The suit is the latest in hundreds of cases filed against social media and AI platforms over alleged harms to young users.

Emma is bringing claims for detective design, failure to warn, negligence, fraudulent concealment and infliction of emotional concealment.

“What happened to Emma is an outrage. Google has no right to use its stranglehold on public schools to force its dangerous Gemini chatbot on a captive audience of millions of kids,” said Julie Liddell, founder of the EdTech Law Center.

Earlier this week, Google faced heavy pushback for announcing students using their software in the classroom will not be permitted to access their search engine unless the school system approves AI use.

This came after Google announced in August it turned on its Gemini chatbot for many K-12 students using Google classroom. Chris Phillips, vice president for education at Google, told The New York Times that “Search and AI are “deeply integrated.”

Several lawmakers called out Google’s move this week, arguing it reflected the company’s monopoly in the online search sector.

Responding to the backlash, Google posted a clarification on X, stating the move was to give “schools more choice, not less.”

By default, Gemini, Gemini Notebook and Search will be off, but schools “can choose to turn on one, two, or all of the services.

“We know there’s concern about students using AI to shortcut schoolwork, and schools have told us they want control,” the company said. “That’s exactly why we are making these changes today to address the feedback we’ve heard from educators in recent months. And we’ll continue to engage deeply with educators to ensure we are giving schools more choice, more controls, and improved Search for learning.”