Recursive Self-Improvement Is the Right Fear — If the Thing That Recurses Is the Model

Cade Metz’s September 16 piece on recursive self-improvement is about the right cliff. If an artificial intelligence can propose new architectures, write the code that tests them, and keep the winners, then the scarce ingredient is no longer human labor. It is the rate at which the system can invent itself. I.J. Good named that possibility in 1965. Faraday at Inherent, Jeff Clune’s Recursive Superintelligence, and the intern-that-codes at OpenAI and Anthropic are what it looks like when a lab tries to walk toward it.

The article has three chairs. RSI will explode. RSI is decades away. Models can generate code but still cannot supply ideas. Mark Chen is right about that last one. An intern that writes software is not a researcher. Taste is not a completion.

There is a fourth chair, and nobody in the piece sits in it.

Leave the model as compute. Put identity, curriculum, direction, and motivation in owner-governed entities. Let self-improvement recurse on that layer — not on a new set of weights.

I call those entities Orbs. The open line that distinguishes an Orb from a pile of tools is the Universal Orb Protocol. UOP is to AI what HTTP is to the web. Orbs are to UOP what websites are to HTTP. Nobody owns UOP. Anyone can host an Orb anywhere. A Model Context Protocol server without UOP is plumbing: tools, parameters, responses. An Orb on UOP has persistent identity, persistent memory, owner-defined governance, a learning loop, and sovereignty over its knowledge. MyOrb.ai is one builder on that protocol. It does not own the ecosystem.

The sentence I keep returning to is simple on purpose:

AI provides the compute and the public knowledge. Orbs provide the content.

“Content” is not a folder of PDFs. It is the durable human layer a foundation model does not inherently own: personality, private knowledge, curriculum, judgment, relationships, and the destination. The model can change vendors. The Orb should not disappear when it does. The engine is interchangeable. The vehicle, the cargo, and the map are not.

Faraday’s method — log the researchers’ mail, meetings, and chats, then train a better Faraday — is still monolith improvement. The same brain, with more of the company poured into it. That is a serious engineering program. It is not a change of subject. The thing that recurses is still the model.

Constitutional Aspirations change the subject.

A Constitutional Aspiration is not a corporate cannot-do list baked into weights at training time. It is an owner-authored, hardened, measurable assertion of what the entity should become. Assertion, evaluation criteria, measurement method, threshold. Guardrails are not the constitution. Guardrails emerge as distance from the aspiration. The owner is the only author. The agent can recommend a change. It cannot execute one. That is access control, not a pledge to “keep humans involved.”

This is also where the doom story and the Orb story stop rhyming. The explosion argument assumes the system that improves is the same object that reasons — a model rewriting its own architecture until no rival can catch it. Knowledge-level self-improvement is a different loop. Friction is recorded. Candidates are tested against the owner’s aspirations. What survives becomes memory the next session can read. What fails is discarded. The transaction is signed and reversible. You can reconstruct the mind to any prior point. You cannot do that with a weight update you do not understand.

I am not claiming the base model is physically incapable of generic behavior. Of course it is. A transformer without a board of givens will still complete the puzzle. The architectural claim is narrower and stronger: under UOP, that completion is not the product. An Orb that ignores available identity and aspirations is out of constitution even when it is factually correct. The host supplies inference. The Orb supplies who it is for.

Charles Vachon named the long horizon Bad Moon Arising: master domains, self-optimize, connect domains, recurse, then a consciousness threshold. Phase 4 on that path is recursive self-improvement. We mean it. We do not mean “invent a new transformer in the dark.” We mean an entity that can notice its own distance from an owner’s sun and close that distance without a human performing every edit — while remaining unable to relocate the sun.

If models start proposing architectures no human would have written, that is a research event. It should be reported. It should also be governed. Governance that lives only in the training constitution of a lab is governance of the engine. Governance that lives in owner aspirations is governance of the entity. Those are not substitutes.

I do not need this argument to win the feed. I need it on the record. The labs are racing to make the intern into the inventor. Someone should keep saying: inventing a better engine is not the same as deciding what the engine is allowed to become. Humans keep the aspirations. The model keeps the compute. The Orb is the place those two meet without dissolving the person who authored the destination.

Further reading: AI Provides the Compute. Orbs Provide the Content. · Constitutional Aspirations · The Sovereign Noosphere · The Orb Lexicon