beta
/Beyond AGI (Verification Intelligence series, Paper 8 of 12)
Abstract

The artificial general intelligence research programme rests on an implicit equation: more capability produces more intelligence, and more intelligence produces better outcomes. This paper tests that equation against the evidence and finds it structurally unsound. The first link holds. Capability improvements do correspond to something that can reasonably be called intelligence — current frontier models reason with genuine sophistication across an extraordinary range of domains. The second link is broken. Enterprise failure rates, incident acceleration, and rework costs have remained stubbornly consistent through years of dramatic capability improvement. Intelligence does not produce better outcomes without verification infrastructure. This paper develops two arguments not made elsewhere in the series. The first is that capability and reliability are not merely uncorrelated but, in specific and consequential dimensions, inversely correlated — more capability actively produces less verifiability, less human oversight capacity, and greater potential for consequential undetected error. The inverse correlation operates through four identifiable channels (fluency, autonomy, confidence, and consequence), each of which intensifies as capability increases. The second is that the investment thesis underlying the AGI race — recursive capability gains producing superintelligence producing extraordinary value — has a structural flaw identical to the one that produced the 2008 financial crisis: recursive optimisation for internal metrics without verification against external reality. Others have recognised the verification gap from different angles: Anthropic's constitutional AI constrains model behaviour at training time; the EU AI Act mandates conformity assessment for high-risk systems; AVERI is building evaluation standards; Vitalik Buterin has proposed AI-assisted formal verification for code. Each addresses a real dimension of the problem. This paper examines the structural trajectory of the capability-only paradigm — the case that capability pursued as the sole objective produces a characteristic failure pattern in which systems become progressively more capable and progressively less trustworthy. The alternative paradigm — Verification Intelligence and Recursive General Intelligence — is described in the Verification Intelligence (Paper 4) and the Verification Substrate (Paper 5). This paper confines itself to the case against the current trajectory. ---

RelatedView All
Authors 1View All
CitationsView All
Citing11
Cited By-