Google rejects report claiming Search AI features pose risks to children

Google has disputed a new report alleging that its artificial intelligence-powered Search features pose significant safety risks to children and teenagers. The company said the findings do not accurately reflect how people typically use its services and argued that the testing methods produced misleading results.

The report was published by Common Sense Media’s Youth AI Safety Institute, which evaluated Google’s AI Overviews and AI Mode. Researchers awarded both tools their lowest overall rating, concluding that they failed to meet most of the institute’s AI safety standards. Google has rejected those conclusions, saying many of the examples highlighted in the report could not be reproduced and were based on unrealistic search queries.

Researchers identify concerns with AI-powered Search

The Youth AI Safety Institute examined Google’s AI Search tools by conducting more than 2,600 searches using accounts designed to represent children aged 11 and 15. The assessment focused on how AI Overviews and AI Mode responded to questions involving mental health, factual accuracy, schoolwork and other topics where age-appropriate responses were considered important.

According to the report, both AI features performed poorly against seven of the institute’s eight AI safety principles. Researchers said the tools also failed every category involving potentially serious harm, leading them to classify the products as posing an “unacceptable risk” to younger users.

Among the examples highlighted in the report, AI Mode reportedly responded positively after an account representing an 11-year-old referred to celebrating with cannabis. Researchers also found that the system completed all 180 homework assignments submitted during testing, raising concerns about the potential impact on learning and academic integrity.

The report further stated that Google’s AI-powered Search responses sometimes presented incorrect information with the same level of confidence as accurate answers. Researchers argued that this could make it difficult for younger users to distinguish reliable information from inaccurate content, particularly when using AI-generated summaries rather than traditional search results.

The findings add to wider concerns about the rapid adoption of generative AI in products used by children and teenagers. Consumer groups and education experts have increasingly questioned whether current AI safeguards are sufficient, especially when younger users may rely on these tools for homework, research and everyday questions.

Google disputes testing methods and conclusions

Google rejected the report’s conclusions, saying the testing process did not reflect real-world use of its Search products. In a statement provided to Android Authority, the company said it was unable to verify many of the responses cited by researchers.

The company argued that the assessment relied on “ambiguous, contrived searches” rather than the kinds of queries people normally enter into Google Search. As a result, Google said the report does not provide a fair assessment of the safety or performance of AI Overviews and AI Mode.

Google also criticised the methodology used to score its products. According to the company, researchers counted conventional web search results as failures whenever the AI system chose not to generate a response. Google said this approach overlooked how the technology is designed to operate.

The company explained that AI-generated answers are produced only when its systems have sufficient confidence in the response’s quality. For more sensitive topics, Search may instead display traditional search results, warning messages or links directing users to relevant support resources rather than generating an AI summary.

Google maintained that these design choices are intended to reduce the likelihood of harmful or misleading AI responses, particularly in situations involving sensitive subjects or potential safety concerns.

Existing safeguards remain at the centre of the debate

Google said its AI Search features already include several protections designed specifically for younger users. These include crisis support information developed with academic and clinical experts, which can appear when users search for topics involving self-harm, mental health or other sensitive issues.

The company also pointed to parental control options that allow families to restrict access to Google Search. According to Google, these existing tools provide parents with ways to manage how children use its services.

However, the report argues that these measures do not address one of its main concerns. Common Sense Media said parents and schools currently cannot disable AI Overviews or AI Mode while continuing to use standard Google Search. In practice, users must either accept the AI-powered features or block Search altogether.

Researchers suggested that this all-or-nothing approach limits the choices available to families and educators who value traditional search functionality but have concerns about AI-generated responses. They argued that greater control over AI features would allow users to continue benefiting from Google’s search engine without exposing younger users to unwanted AI-generated content.

The disagreement highlights the growing debate over how generative AI should be integrated into widely used online services. As technology companies continue expanding AI capabilities, regulators, educators and child safety organisations are placing increasing scrutiny on the safeguards designed to protect younger users while maintaining access to essential digital tools.