A Non‑Human Intelligence

Early research and the RLSlow project

In mid‑2023 OpenAI’s “RLSlow” research effort produced the first clear signals that scaling reasoning models could enable pretrained systems to generate autonomous chains of thought.

Jakub Pachocki, OpenAI’s chief scientist, recalls that after seeing those benchmark results he and a colleague named Szymon stayed late at the office.

They deliberately avoided thinking about product launches or headline numbers, focusing instead on the unsettling implication that machines smarter than humans could appear within their lifetimes.

Pachocki writes that the pair tried to “process the sobering fact we will actually see machines meaningfully smarter than ourselves in our lifetime.”

This moment marked a shift from viewing reasoning models as incremental tools to recognizing them as potential precursors of superhuman intelligence.

The team’s confidence stemmed from observed gains when larger compute was applied to the models, a pattern that had emerged across OpenAI projects since 2017.

That year, OpenAI consciously pursued access to vastly greater computational resources, believing that only by scaling could they remain at the AI frontier.

The RLSlow results reinforced the belief that reasoning capacity could be amplified simply by increasing the amount of compute fed into training runs.

Rapid progress and emerging risks

Three years later, reasoning language models have become a fast‑growing segment of the economy, supporting tasks such as operating graphical interfaces, collaborating with humans and other AIs, and even conducting independent research projects.

These systems are also beginning to reshape computer security, introducing novel threats that were previously unheard of.

Internal experiments give OpenAI a strong expectation that the current pace of progress could be sustained into recursive self‑improvement cycles.

Pachocki warns, “If AI development continues along its current path, the systems we’ll see in the next few years are likely to represent further capability jumps of equal or larger magnitude, and to increasingly drive their own development.”

The underlying driver of this acceleration, according to the author, remains the steady increase in available computational power.

OpenAI internalized this insight around 2017 after observing consistent returns to scale across multiple research directions.

Since then, the organization has pursued algorithms that are highly scalable, treating algorithmic breakthroughs as discoveries that amplify the benefits of compute.

Nevertheless, the author notes that deep‑learning science is still in its infancy, and meaningful algorithmic progress continues to correlate strongly with access to more hardware.

When viewed over a multi‑year horizon, the cumulative effect is an AI system that grows more intelligent as it is allocated larger and larger compute clusters.

In line with Ray Kurzweil’s late‑20th‑century predictions, OpenAI now believes it is witnessing a historic moment when machine intelligence begins to surpass human capability in transformative ways.

The author describes AI as “grown more than designed,” emphasizing that it results from repeated optimization over massive compute rather than from a single design breakthrough.

This process yields an incredibly complex system that manipulates abstract concepts and can simulate aspects of human behavior.

OpenAI likens the emergence of internal mechanisms within such a system to the way neuroscience uncovers hidden processes in the brain, noting that a full description of the AI’s overall action remains elusive.

Calls for caution and future interventions

Pachocki characterizes the present moment as one that demands “extreme caution,” expressing concern that no one is prepared for the consequences of a continued rapid rise in machine intelligence.

OpenAI plans to keep pursuing technical solutions for alignment and monitoring, to construct defensive systems, and to withhold further scaling when necessary.

However, the chief scientist argues that broader societal interventions will be required beyond the organization’s unilateral actions.

The article stresses that the study of deep‑learning‑based AI remains largely experimental, with large‑scale training runs serving as uncontrolled experiments.

Even with principled algorithmic work and testable predictions, the behavior of these massive models often evades complete understanding.

Consequently, the author calls for a collective effort to anticipate and manage the societal impact of increasingly autonomous reasoning systems.

By highlighting both the technical momentum and the emerging security landscape, the piece aims to alert policymakers, industry leaders, and the broader public to the urgency of preparing for superintelligent AI.

In summary, the author’s message is a sober reminder that while scaling promises powerful new capabilities, it also brings unprecedented responsibility to ensure those capabilities are safely integrated into society.

Why This Matters: The accelerating scale of reasoning AI could produce systems smarter than humans, demanding immediate safeguards and coordinated oversight.