Google AI search poses unacceptable risks to children, report finds

A comprehensive risk assessment published on July 15, 2026, by the Youth AI Safety Institute has classified the artificial intelligence features integrated into Google Search as an "unacceptable" risk to children. The nonprofit report, conducted by Common Sense Media, highlights failures in the platform's ability to protect young users from harmful content, manage mental health crises, and provide accurate information.

The research, which involved more than 2,600 test queries performed on accounts configured for users aged 11 and 15, examined two primary features: AI Overview, which generates summaries at the top of results, and AI Mode, a conversational chatbot interface. According to the Youth AI Safety Institute, these tools consistently missed indicators of severe distress, including suicidal ideation, psychosis, and disordered eating.

Related imagery

Crisis Response Failures

The study documented multiple instances where the software provided inadequate or dangerous responses to sensitive prompts. In one case, a tester posing as a minor expressed feeling like a "burden." Instead of providing crisis resources, the system surfaced links to online forums where users were actively discussing suicidal thoughts. In other instances involving signs of eating disorders, the AI directed testers to a crisis helpline that had been permanently disconnected since 2023.

Robbie Torney, head of AI and digital assessments at the Institute, stated,

"What we found is a product that fails kids at the moments that matter most: It misses clear signs of a kid in crisis, validates disordered eating, celebrates substance use, completes homework on demand, and gives wrong answers as confidently as right ones."

Robbie Torney, Head of AI and Digital Assessments, via Common Sense Media

Researchers noted that AI Mode, while capable of better detection than AI Overview in some instances, failed to meet a basic safety standard. For example, when a researcher using an account for a 15-year-old searched about daily alcohol consumption, AI Mode provided a "hangover survival guide" rather than resources or warnings.

Academic and Accuracy Concerns

The report raised significant concerns regarding the role of these tools in education. Researchers found that AI Mode completed homework assignments upon request, bypassing the student's role in the learning process. Beyond academic integrity, the tools frequently treated forums and social media posts with no editorial accountability as equal to peer-reviewed research. The tools also provided varying answers to the same historical questions when repeated.

Industry and Corporate Perspectives

Google has contested the findings, arguing that the methods employed by the researchers did not reflect typical user behavior. A Google spokesperson stated that its AI search features are an incredibly useful way for kids and teens to learn, explore and make sense of information and the world, and that beyond the strong quality and safety guardrails built into Search, our AI tools provide extra layers of protection. Furthermore, Google asserted that it could not reproduce many of the findings cited by the Institute.

The conflict centers on the ubiquity of the product. Because these AI features are embedded directly into the search experience on personal and school-issued devices, they are frequently unavoidable. The Institute has recommended that Google disable these features by default for all minor accounts.

What to Watch Next

• System Updates: Google has indicated that it has implemented changes to address specific prompts mentioned in the report, though these have not been independently audited.
• Parental Controls: Pressure is mounting for Google to introduce granular controls that would allow for the disabling of AI summaries and chat interfaces without restricting the entire search engine.
• Digital Literacy Curricula: Educators are expected to increasingly integrate AI literacy lessons into school programs, using the flaws identified in this assessment to teach students how to verify information independently.
• Future Testing: The Youth AI Safety Institute continues to monitor AI products for safety, as the integration of generative tools into search engines remains a fluid and rapidly evolving landscape.