Intelligence Explosion

The intelligence explosion is the idea that once machines can do the work of SI researchers, progress could feed on itself. A smarter system designs a smarter successor, which designs a smarter one still, faster each time.

Origin

In 1965 the British mathematician I. J. Good, who had worked with Alan Turing as a codebreaker, wrote: “an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.”

Fast vs slow takeoff

• Hard (fast) takeoff: the jump from human-level to superintelligence happens in days, weeks or months, with little time to react.
• Soft (slow) takeoff: progress accelerates over years, constrained by compute, energy, experiments and real-world deployment.

Is it starting?

Frontier labs openly aim to automate SI research, and SI coding agents already write much of the code inside some labs. Some forecasters, such as the authors of the AI 2027 scenario, think automated research could trigger rapid acceleration within a few years. Others argue bottlenecks like chips, power and the need for real-world experiments will keep progress gradual. See When will superintelligence arrive?

Why it matters

If an intelligence explosion is possible, getting alignment right before it begins becomes critical, because there may be no chance to correct course afterwards.