Microsoft CEO: The era of superintelligence requires building a "controllable" architecture, and responsibility cannot be outsourced to AI

Odaily News: Microsoft CEO Satya Nadella published a long article stating that with the development of Super Intelligence systems, the industry needs to re-examine AI trust architecture, and cannot simply rely on commitments from model providers, nor outsource the responsibility brought by AI behavior.

Nadella said that when traditional software systems were deployed across industries over the past few decades, developers could usually trace the specific code paths corresponding to behavior. But today's frontier AI model capabilities have surpassed traditional software systems, yet it is impossible to clearly attribute model outputs to specific training data or model weight configurations. Enterprises are connecting AI systems with autonomous action capabilities to sensitive data and granting them permissions to perform critical tasks, so they need to build systems with observability, testability, and controllability, separating "intelligence supply" from "permission control."

Nadella proposed that superintelligence should be governed through an engineering approach, constraining nondeterministic models through deterministic system design, human control, and operational processes, and treating AI systems as potential "insider risks" for management, including limiting permissions, logging behavior, and establishing isolation boundaries. He also proposed that superintelligence systems should follow principles such as multi-model collaboration, end-to-end observability, continuous verification, independent control, independent auditing, risk isolation, and incident disclosure. Among them, he emphasized that transparency of the model chain of thought (CoT) is a basic requirement, but CoT transparency alone is not enough, because the model output itself may still lack reliable interpretability.

Nadella finally said that the most trustworthy superintelligence system in the future is not the "most trustworthy model," but a "system that can still operate safely even if the model is not fully trusted."