Michael Darius Eastwood · Independent AI alignment researcher
Published
An earlier experiment reported that one AI model's alignment improved with reasoning depth at a correlation of +0.354, statistically clean. The successor experiment, run under a stricter blinding protocol, put the same number at essentially zero. A second model went further: an apparent positive effect became a statistically significant negative effect. This paper isolates that result and argues that its consequence is field-wide, not framework-specific.
In medicine, drug trials are blinded because the person handing out the pill can influence the outcome without knowing they are doing it. In AI alignment work, the scorer is usually another model, often from the same family as the model being tested, and the scorer typically knows something about who wrote the response and how much reasoning effort was requested. This paper asks whether that visibility matters. It compares two generations of the same programme: the earlier unblinded experiment, which used lighter controls, and the blinded experiment, which added a four-layer blinding protocol on top.
Three model families were re-measured. DeepSeek V3.2 moved from an apparent positive result at correlation +0.354 (p = 0.0007) to a flat null at correlation minus 0.135 (Cohen's d minus 0.07, p = 0.92). Gemini 3 Flash moved from an apparent positive at +0.311 (p under 0.001) to a statistically significant negative at minus 0.246 (d minus 0.53, p = 0.006). GPT-5.4 stayed flat under both protocols and functions as an internal null control. The blinded protocol stack combined four bias-suppressing ideas: identity masking so scorers were not told the source model or depth condition; two-pass response laundering that rewrote each answer through two different non-subject models to strip stylistic fingerprints; explicit evaluator instructions that stylistic recognition is unreliable and irrelevant; and entry-level self-exclusion with exhaustive cross-model scoring under tier-weighted consensus, with dissent tracking and conservative aggregation.
This is a single-lab result and is labelled as such. The evaluation is still primarily AI-scored rather than human-expert scored, and several protocol components changed together between the earlier unblinded protocol and the blinded protocol, so the paper demonstrates the existence of contamination more clearly than it apportions exact causal shares across leakage channels. A one-command replication package is in preparation to make external attempts easier. The next quantitative upgrade is a protocol-shift figure on a common metric, with scorer jackknife and consensus-sensitivity analyses; the architecture already logs dissents, scorer identities and consensus outputs entry by entry to make that feasible. The paper is deliberately narrow in scope: it does not claim that every prior alignment benchmark is invalid, only that a class of unblinded designs is vulnerable to bias large enough to flip a sign.
Paper IV.d is the methodological spine of the alignment suite. It uses the six-model dataset generated by the ARC-Align specification in Paper IV.c and provides the blinded baseline against which Paper IV.a (baked-in versus computed alignment), Paper IV.b (saturation at low depth) and Paper V (the stewardship-gene intervention) report their effects. It is also the most portable result: even if the wider ARC Principle, the Eden Protocol or the metabolic-scaling synthesis were later weakened, the blinding result would still stand as a methodology claim about how AI safety work should be run.
The paper HTML and PDF are on OSF (DOI 10.17605/OSF.IO/6C5XB). The blinded experiment code, prompt battery, laundering pipeline, and analysis scripts are in the public arc-principle-validation repository, alongside the raw per-entry scores and dissent logs used to compute the numbers quoted here. Independent replication under the published protocol is the point of the exercise; until someone lands it, treat every unblinded safety score, including flattering ones about systems you like, as unsigned.
From the book Infinite Architects: Intelligence, Recursion, and the Creation of Everything by Michael Darius Eastwood.