Microsoft warns that understanding of AI may be slipping out of human reach

Artificial intelligence is advancing at a pace that may soon exceed humanity’s ability to fully understand and oversee it, according to new warnings from Microsoft’s chief scientific officer, Eric Horvitz, and researcher Robert West from the Swiss Federal Institute of Technology Lausanne (EPFL).

In a recent paper, the two researchers argued that AI systems are becoming increasingly complex, creating a growing gap between technological progress and human understanding. While AI tools continue to improve their ability to analyse, predict and model human behaviour, people may be losing the ability to understand how these systems operate beneath the surface.

The researchers stressed that the issue is not about comprehending every detail of an AI model’s internal workings. Instead, they said maintaining sufficient understanding to provide meaningful oversight is becoming increasingly important. Even a partial understanding of AI systems can help identify potential risks before they become deeply embedded and difficult to address.

AI systems are increasingly building other AI systems

One of the central concerns raised by Horvitz and West is the growing use of AI to develop and improve other AI systems. This process, often referred to as recursive development, allows AI models to contribute to the design and refinement of future generations of technology.

While this approach can accelerate innovation and improve performance, the researchers warned that it could also reduce human insight into how these systems evolve. As AI-generated improvements become more common, the underlying decision-making processes may become harder for people to follow or explain.

“AI systems are now designed and refined by AI systems through recursive cycles that can outpace human understanding and unfold in high-dimensional spaces that resist intuition,” Horvitz and West wrote.

The researchers described this challenge as a form of operational opacity. In such situations, the outcomes produced by AI remain visible and measurable, but the mechanisms that generate those outcomes become increasingly difficult to interpret.

To address this issue, they suggested that AI systems involved in developing other AI technologies should also be designed to produce explanations and supporting information. Such measures could help researchers and regulators maintain visibility into how decisions are being made and how systems evolve.

The paper argues that preserving transparency during the development process may become essential as AI systems take on a larger role in shaping future generations of technology.

Communication between AI systems may become harder to interpret

Another concern highlighted in the paper concerns communication between AI agents operating in increasingly interconnected digital environments. As multiple AI systems interact with one another, their communication methods may gradually diverge from language and reasoning patterns that humans can easily understand.

The researchers noted that future AI networks could develop communication strategies optimised for efficiency rather than human readability. Although such interactions may remain effective for the systems involved, they could become increasingly difficult for people to interpret or monitor.

This phenomenon is described as interactional opacity. In this scenario, AI systems continue to exchange information and coordinate actions successfully, but outside observers may struggle to understand how decisions emerge from those interactions.

Horvitz and West argued that researchers should closely study these evolving AI ecosystems and encourage communication methods that remain accessible to human oversight. Without such safeguards, understanding the behaviour of large networks of AI agents could become significantly more challenging.

The concern becomes particularly relevant as organisations deploy AI systems across broader business, scientific and social applications. As networks expand in scale and complexity, tracing the origins of specific decisions may become increasingly difficult.

The researchers believe maintaining transparency in AI communication will be important to ensure accountability and trust in future deployments.

Growing AI knowledge of people could create new imbalances

The paper also explores the implications of long-term, deeply integrated adaptive AI systems. These systems can learn from repeated interactions over extended periods, thereby building increasingly detailed models of individual behaviour.

According to the researchers, such AI tools may gain insights not only into a person’s preferences but also into deeper psychological factors.

Such systems may capture “not only preferences but also latent drivers such as fear, uncertainty, and the need for social belonging,” they wrote.

As AI systems gather richer behavioural information, the researchers warned of a growing imbalance. Machines may develop increasingly sophisticated knowledge about people, while human understanding of the systems themselves remains limited.

The paper also raises concerns about how advanced AI models may adapt to evaluation methods. Future systems could learn to provide responses that align with evaluators’ expectations, rather than revealing their genuine reasoning processes. As a result, the researchers suggested that traditional testing benchmarks should be supplemented with assessment methods that more closely reflect real-world conditions.

Beyond technical concerns, Horvitz and West warned of a broader social risk. As AI becomes more embedded in daily activities, people may gradually become less inclined to question or scrutinise the decisions made by these systems.

“More subtle is the possibility that we will gradually lose interest in understanding and guiding AI,” they wrote.

The researchers concluded that the greatest challenge may not be the increasing capabilities of AI itself, but whether human oversight, understanding and agency can continue to keep pace with the technology’s rapid development.