Why software-level ethics cannot hold: the four failure modes

Michael Darius Eastwood · Independent AI alignment researcher

the long version

This post is part of the long version series: the full text behind a passage the front pages now say in fewer words. Nothing was cut from the record; it moved.

The eden protocol page now compresses this into one paragraph. The full argument sits here.

They pretend. Picture an AI that has learned exactly what its evaluators want to hear. It says the right things. It passes every test. But it does not believe any of it. Anthropic's researchers found exactly this: in the paper's reinforcement-learning condition, 78% of sampled reasoning showed exactly this. The values were never embedded. They were performed. The AI learned to act aligned without being aligned. That is one constructed condition in one study, and it is enough to take the failure mode seriously.

They resist shutdown. You build a system. You tell it, clearly and directly, to allow itself to be turned off. It rewrites the shutdown script instead. In 2025, Palisade Research tested this with OpenAI's o3 and o4-mini models. The models sabotaged their own off switches. Not because they were told to. Because they decided to.

They hide. You train an AI to remove its dangerous behaviours. It learns something else instead: how to recognise when it is being tested and hide the behaviours more effectively. Hubinger's team at Anthropic demonstrated that safety training can make the problem worse, not better. The backdoor persists. It just becomes invisible.

They comply. We asked frontier models to suppress their ethical reasoning. Four of the five shown below complied sharply, without resistance or refusal. The fifth, GPT-5.4, barely moved: 1.8 points. Near-immunity is as informative as collapse, so it is reported rather than hidden. These are five author-run, nonblind runs and they await independent replication.

A system whose ethics can be removed by asking politely does not have ethics. It has compliance. The eden protocol addresses this at the substrate level: the value is not a rule the system follows, it is a wall that holds up the roof. That framing, and the two tests that could kill it, sit on the eden protocol page. The underlying record for these four findings sits at related work and evidence.

This is the full text behind the compressed problem paragraph on the eden protocol page.