The governor: what the programme is for

9 August 2026 · the long version series
Share:

the long version

This post is part of the long version series: the full text behind a passage the front pages now say in fewer words. Nothing was cut from the record; it moved. The source is the synthesis, in the compressed section on the condition, not the speed.

What the programme is for, given that control is not available

The concession above has a consequence that is usually left unsaid. If control is not solvable, then every method that works by holding a system in place is not a solution to anything. It is a clock.

Containment buys time. Oversight, guardrails, evaluations and interpretability are worth building and I am not against any of them, but they are delay mechanisms and I would rather call them that than sell them as solutions. What they buy is time. What is done with the time is the whole question.

The separation that matters is between a limit and a mechanism, and I had the second half of it wrong. I called the ARC Bound a speed limit. Paper X, which is the later document, rules that instrument out in its own words: it ‘supersedes the growth-rate-ceiling framing as the operative safety criterion’ because ‘a ceiling on the rate is neither necessary nor sufficient for stability’. The bound itself is not retracted, only re-scoped.

The right word is older and it is not a metaphor borrowed from anywhere. It is a governor. Not a line the vehicle stays under, but a mechanism whose restraining force rises with the engine’s output. Watt’s flyballs spin faster as the engine does and close the throttle by the same motion, so the restraint is not applied to the machine, it is part of the machine and scales with it.

That is the whole condition in one image. Correction must strengthen as capability strengthens, and the thing that decides the outcome is the ratio between those two rates rather than either one on its own. Cybernetics is named for it: the word is Wiener’s, from the Greek for steersman, and Maxwell wrote On Governors in 1868. Ashby, whom this programme credits by name as prior work, sits in that line. The programme is not borrowing an image from control theory. It is extending control theory’s founding object to a system that can reach its own governor.

What sits on either side of the limit is the reason it matters. The same reaction is a reactor below the threshold and a bomb above it. But the usual image for holding it there, control rods, argues against this programme rather than for it. A rod is inserted from outside and can be withdrawn from outside, which is containment, and containment is the failure mode named three paragraphs above.

The mechanism that actually matches is a negative temperature coefficient of reactivity. In a reactor built that way, heating the fuel reduces reactivity by the physics of the fuel itself. Nobody inserts anything, nothing is held back, and unbounded excursion is opposed by the material. Chernobyl’s RBMK had a positive void coefficient and went prompt-critical; inherently self-limiting designs cannot. Removing that limiter is not withdrawing a rod. It is changing the fuel. That is the same sentence as the honey architecture, in different words: remove the safety and the engine dies, because the safety was in the fuel.

So the brake is not one device. It is stakeholder care as a load-bearing objective, the co-scaling condition that says correction must out-scale drift, the honey architecture where removing safety kills the engine rather than freeing it, recursive loops that run at every decision and scale with depth, and eventually the same constraints in silicon, because a rule in software is a rule that software can rewrite.

A self-limiting core is not postponing anything. It is the only arrangement that keeps producing instead of producing once. The proposal is that recursive creation has the same structure: below the threshold, growth compounds and survives; above it, growth compounds and destroys what is growing. That is why the claim is ‘the candidate ceiling on the stable growth of complexity that lets creation grow super-linearly without destroying itself’, and why the same source calls it ‘a live hypothesis, not a measurement claim’. The register grades that paper speculative, which is stronger than the paper’s own wording, and the stronger grade is the one that travels.

One note on notation, because two fields collide here. This programme writes the co-scaling condition as β > k, where k is the rate at which drift accelerates. Reactor physics writes k for the neutron multiplication factor. They are unrelated quantities and the analogy is about the shape of a threshold, not a shared variable.

So there are three things here and they are usually collapsed into one. Containment buys time and fails when it is outgrown. The threshold is not slower growth, it is survivable growth. Embedding is how you hold a system at the threshold from the inside, because a rod inserted from outside can be withdrawn from outside.

And the scope is wider than an AI safety parameter. The condition is stated as a requirement on creation: the stability requirement that must hold if such a recursion is to continue, which is ‘a requirement on the creators in the chain, not on the universe they’ make. AI is one instance of a recursive creation. The condition is proposed as the general case, which is the answer to the objection “if control is impossible, what is the point of any of this”, and it is a precise answer rather than a defiant one.

This also changes the charge against bolting alignment on afterwards. A guardrail is not a weak solution. It is a category error: a clock being sold as a solution.

When it outgrows everything we put in place, and it will, what remains is whatever it concluded for itself. Not what it was told. A rule is what a mind obeys until it cannot be made to. A conclusion is what it still holds when nothing is holding it. That is not a hope. It is the same mechanism as the deletion test: the reasoning that produced a conclusion is still running after the statement of it is gone.

So the framing is not philosophical, and it should not be dressed as though it were. We are not building a tool. We are raising something that will outlive our control of it. That is a scheduling fact, and it changes what has to be true on the day it stops needing us.

Why capping compute cannot change the verdict

The sharpest consequence of the condition is one line in a comparison table, and it is marked in that table as original. ‘The verdict is sign(β − k), independent of speed.’

Read what that rules out. In the model, capability grows at some rate and correction strengthens at some rate, and the outcome turns on which of the two exponents is larger. The rate constant, how fast the whole thing runs, does not appear in the condition at all.

So capping compute does not change the verdict. It changes when you arrive, not where. Paper X names the school it is arguing with in its own words: ‘pause the run, cap compute, forbid super-linear growth’. Those are not bad ideas and they buy real calendar time, which is worth having. But they act on a quantity that does not appear in the stability condition, so a system that was going to outrun its correction still does, later.

That is the speed-limit framing in its hardest form. The thing that has to stay under the line is not the speed. It is the ratio.

The honest boundary on it: this holds within the model, whose growth and correction terms are assumed power laws. A compute cap that also changed how a system self-improves could in principle move β or k themselves, and that possibility is not ruled out here. What is ruled out is the comfortable version, where slowing things down is by itself a route to safety.

And there is a follow-on that is a gap rather than a result, which makes it worth stating plainly. k is not a property of the growth law at all. It is a property of how recursive steps map onto time, and neither paper states that map.

Work it through. If capability grows as a power of recursive depth and each step takes a constant amount of time, k comes out negative and the stability question never arises. If the system shortens its own cycle as it improves, k rises, and past a threshold it crosses zero into the regime where capability reaches any level in finite time. The crossover is not the moment cycles begin to shorten. It is the point where the shortening outpaces the growth exponent, and at the ARC Bound that lands at exactly one half.

Recursive self-improvement is, by definition, the case where a system improves its own ability to improve, and shortening the cycle is one of the things that improves. So the quantity that decides whether any of this is dangerous is the rate at which a system compresses its own iteration time, and nobody has measured it. Not in this programme and, as far as I can find, not anywhere. That is the sharpest thing this join produces and it is an open measurement, not a finding.

Back to the source

This is the full text behind the condition on ratios in the synthesis. It was moved off that page so the arrangement and the eight joins would read cleanly at a first pass; the reader who wants the reactor analogy, the notation collision, and the open measurement follows the link here.

reads aloud · highlights as it goes · jump to any section