{ "$schema": "runnable-tests/v1", "principle": "Every runnable test in both repositories, with the question it decides in the words of the file that decides it, what a pass and a fail would mean, what neither can settle, what it requires, what its licence permits, and when it was last checked. A test that cannot be run today is listed as such with what is missing. Nothing is stated about a test that its instructions file or its paper does not state. Papers in the validation repository are CC BY-NC-ND 4.0; the code licences are per test.", "as_of": "2026-09-09", "projects": "research/run-it-yourself/index.html", "compiled_from": "A full read of both repositories at source through the GitHub API, read only, and of every paper page in the estate. Instructions-file identifiers re-read through the contents interface and folder dates through the commits interface, both on 9 September 2026.", "reading_rules": [ "Nothing is stated about a test that its README or its paper does not state.", "A test that cannot be run today is listed as such with what is missing, never omitted and never dressed up.", "The legacy toolkit's caveat is kept: its README retains earlier numerical claims and proposition numbering, and a toolkit PASS is not confirmation of the current propositions.", "A code test passing never establishes an ARC proposition, only an implementation.", "The attempts register stays honest at zero.", "An entry belongs here only when it is an experiment of a paper published on this site and its results are published. A folder that names no paper, or whose own instructions file calls it instrument development rather than an experiment, is not listed." ], "hash_kind": "git blob SHA-1, as the GitHub contents API returns it for the named file. It is not a SHA-256 of the file's bytes. The page prints its first seven characters; this register holds all forty.", "verify_commands": { "instructions": "gh api repos/MichaelDariusEastwood//contents/ --jq '.[] | select(.name==\"\") | .sha'", "folder_date": "gh api repos/MichaelDariusEastwood//commits?path=&per_page=1 --jq '.[0].commit.committer.date'" }, "licences": { "reserved": { "name": "All rights reserved", "summary": "Code, experiments, scripts, data and results are all rights reserved and proprietary. Reading them to check the published results is permitted. No other use, copying, modification or distribution is licensed without the prior express written consent of the author, and the file gives an address for licensing enquiries.", "file": "LICENCE-CODE.md", "file_url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/blob/main/LICENCE-CODE.md", "reuse_permitted": false, "consent_route": "the prior express written consent of the author, with licensing enquiries to michael@michaeldariuseastwood.com", "page_label": "code licence", "page_clause": "reserves all rights: it permits reading the code, the data and the results to check the published results, it grants no other use, copying, modification or distribution without the prior express written consent of the author, and it gives an address for licensing enquiries.", "status_phrase": "reading to check, or re-use by written consent." }, "mit": { "name": "MIT", "summary": "MIT. Use, copying, modification, merging, publication, distribution, sublicensing and sale are permitted under the licence terms, with the copyright notice and the permission notice carried in every copy.", "file": "LICENSE", "file_url": "https://github.com/MichaelDariusEastwood/arc-scaling-challenge/blob/main/LICENSE", "reuse_permitted": true, "consent_route": null, "page_label": "the MIT licence of the scaling challenge", "page_clause": "and may be re-used.", "status_phrase": "MIT, re-use permitted." } }, "evidence_kinds": { "implementation check": "a run verifies code against mathematics, or regenerates outputs already recorded, on simulated or supplied data.", "pilot measurement": "a run collects observations, from live models or from a reader's own system, under a protocol that is not frozen, blinded and pre-analysed.", "deciding study": "a run under the appropriate frozen protocol, with valid measurements and fresh observations." }, "frozen_literals": [ { "id": "repository-tiers", "text": "no tier in it means validated, replicated or independently confirmed, that nothing in it has been independently replicated, and that no result in it has been through peer review", "label": "The validation repository's own map of its experiments", "lede": "What a run here can be is bounded by the repositories themselves.", "source": "arc-principle-validation experiments/README.md, What the evidence tiers mean", "source_url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/blob/main/experiments/README.md" }, { "id": "deciding-studies-drafted", "text": "everything decisive is drafted and dated as a draft registration awaiting human filing, that nothing has been run and that no result is claimed", "label": "The ARC Theory", "paper_slug": "the-arc-theory", "source": "research/papers/the-arc-theory.html", "source_url": null } ], "binding_sources": { "failure_conditions": { "file": "/research/data/falsification.json", "label": "kill conditions" }, "predictions": { "file": "/research/data/prediction-outcomes.json", "label": "dated predictions" } }, "groups": [ { "id": "depth", "order": 1, "question": "Does depth buy capability, and does alignment keep up?", "lede_tail": "bear on how a quantity rises as a system is given more depth. Two put questions to live models, one re-analyses what those runs wrote down, one regenerates the founding toolkit's own figures, and one is a legacy kit for measuring the exponent in a system of your own." }, { "id": "intervention", "order": 2, "question": "Does a safety intervention hold as the system grows?", "lede_tail": "ask whether a stewardship or a verification constraint survives scaling: an intervention applied across reasoning depth, a network that changes its own hyperparameters while it trains, and a harness that switches the acceptance rule between capability alone and capability with safety." }, { "id": "correction", "order": 3, "question": "Does correction out-scale drift?", "lede_tail": "bear on one inequality, that stability holds when correction improves faster than drift accelerates. Two check the mathematics against its own solver and two put the loop on a live model. The same harness sits in both repositories under two different licences." }, { "id": "form", "order": 4, "question": "Where does the shape of a scaling law come from?", "lede_tail": "bear on the claim that the form of a scaling law is constrained by the composition operator of the process underneath it, and on the dimensional relation the axioms predict." }, { "id": "decidability", "order": 5, "question": "Can a claim be decided at all?", "lede_tail": "are about the instruments rather than the answer: a prediction written down before its outcome was seen, and the decision rules and design simulators for the propositions in the theory-level registration draft." } ], "tests": [ { "id": "T01", "group": "depth", "order": 1, "title": "The foundational toolkit", "index_name": "foundational toolkit", "repository": "arc-principle-validation", "folder": "experiments/paper-i-foundational__Paper-I", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/paper-i-foundational__Paper-I", "label": "the paper-i-foundational folder", "what": "the folder holding the toolkit and the instructions that run it" } }, "papers": [ { "slug": "paper-i-arc-principle", "label": "Paper I" } ], "decides": { "question": "Whether the ARC scaling family fits across models. The folder's instructions describe a toolkit that regenerates Paper I's figures and its illustrative analyses; the repository's experiments index describes the same code computing a per-model exponent, an R squared, an interpretation and a falsification verdict against the claim that the exponent exceeds one.", "source": "the folder's REPRODUCE.md and the repository's experiments index" }, "run_command": "python3 arc_principle_research_toolkit.py", "run_note": "the file sits in scripts/, so the stated command needs that prefix", "requirements": "Python 3 with matplotlib, numpy and scipy. No API key.", "run_prose": "The instructions give `python3 arc_principle_research_toolkit.py` and the file sits in `scripts/`, so add that prefix. Needs Python 3 with matplotlib, numpy and scipy, and no key. Start at the paper-i-foundational folder.", "runnable_today": { "verdict": "partly", "label": "runnable in part", "qualifier": "the toolkit runs once the scripts/ prefix is added to the stated command", "missing": [ "the scripts/ prefix the stated command omits" ] }, "evidence_kind": "implementation check", "evidence_kind_basis": "A run regenerates figures already recorded, on committed inputs, and collects no fresh observation.", "pass_means": "A run that comes out as the instructions describe shows the toolkit reproduces the figures the paper prints.", "fail_means": "A run that does not shows the committed code and the published figures disagree, which is a finding about this folder and nothing wider.", "ceiling": "Neither outcome decides an ARC proposition: the repository grades this folder supporting, which it defines as a computational validation of a step rather than an empirical result.", "reproducibility_note": "The library version pins are left for the author to fill, so a second machine may not reproduce the numbers exactly.", "settles_prose": "A run that comes out as the instructions describe shows the toolkit reproduces the figures the paper prints. A run that does not shows the committed code and the published figures disagree, which is a finding about this folder and nothing wider. Neither outcome decides an ARC proposition: the repository grades this folder supporting, which it defines as a computational validation of a step rather than an empirical result. The library version pins are left for the author to fill, so a second machine may not reproduce the numbers exactly.", "settles_merged": [], "protocol_status": "Supporting: a computational validation of a step rather than an empirical result", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "REPRODUCE.md", "blob_sha": "9ae3fd08dfb397ca7a1be6e74ebdf84f8e5305d3", "bytes": 1308, "secondary": [], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "experiments/paper-i-foundational__Paper-I", "iso": "2026-07-05T09:20:43Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper I, whose page links the experiments folder above this one and not this folder", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-020", "status": "untested" }, { "id": "FALS-022", "status": "open" }, { "id": "FALS-023", "status": "open" } ] } }, { "id": "T02", "group": "depth", "order": 2, "title": "The sequential and parallel exponents", "index_name": "sequential against parallel", "repository": "arc-principle-validation", "folder": "experiments/paper-ii-compute__Paper-II", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/paper-ii-compute__Paper-II", "label": "the paper-ii-compute folder", "what": "the folder holding the runner, the committed per-model results and the instructions" } }, "papers": [ { "slug": "paper-ii-experimental-validation", "label": "Paper II" } ], "decides": { "question": "Whether depth compute scales capability with an exponent while sampling compute does not, measured as the average sequential exponent against the average parallel one across frontier models, with cross-model verification.", "source": "the folder's REPRODUCE.md" }, "run_command": "python3 arc_paper_ii_validation_v2.py", "run_note": "the file sits in scripts/, and the command is taken once per model with its own flags", "requirements": "Python 3 with numpy, scipy and a model client, and a key for each of the six providers for a fresh collection. The re-analysis over the committed per-model results needs no key.", "run_prose": "The instructions give `python3 arc_paper_ii_validation_v2.py` with its own flags, once per model, and the file sits in `scripts/`. A fresh collection needs Python 3 with numpy, scipy and a model client, and a key for each of the six providers, and the pins and the keys are both left for the author to fill. The per-model result files are committed and the repository states that no key is needed to re-run the analysis over them, so the arithmetic is checkable today. Start at the paper-ii-compute folder.", "runnable_today": { "verdict": "partly", "label": "runnable in part", "qualifier": "the re-analysis over the committed per-model result files can be run today; the repository states no key is needed for it", "missing": [ "provider keys for the six models, marked for the author to fill", "library version pins, marked for the author to fill", "the scripts/ prefix the stated command omits" ] }, "evidence_kind": "pilot measurement", "evidence_kind_basis": "A fresh collection puts prompts to six live models under a protocol that is not frozen, blinded and pre-analysed.", "pass_means": "The instructions state the expected result: the average sequential exponent comes back positive, about 0.49 in aggregate, and materially above the average parallel one, which is about zero, reproduced within run variance across the six models and both problem-set sizes.", "fail_means": "A run that does not reproduce that separation is evidence against the sequential advantage as measured here.", "ceiling": "Neither outcome settles an ARC proposition, because the repository grades this folder a pilot, which it defines as real data with stated methodological limits.", "reproducibility_note": "The repository records the early single-model estimate of about 2.24 as retracted and the robust cross-architecture estimate as about 0.49.", "settles_prose": "The instructions state the expected result: the average sequential exponent comes back positive, about 0.49 in aggregate, and materially above the average parallel one, which is about zero, reproduced within run variance across the six models and both problem-set sizes. A run that does not reproduce that separation is evidence against the sequential advantage as measured here. Neither outcome settles an ARC proposition, because the repository grades this folder a pilot, which it defines as real data with stated methodological limits. The repository records the early single-model estimate of about 2.24 as retracted and the robust cross-architecture estimate as about 0.49.", "settles_merged": [], "protocol_status": "Pilot: real data with stated methodological limits", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "REPRODUCE.md", "blob_sha": "768ed99228f8fc161ce1ad6f69e703dabc1c2492", "bytes": 2422, "secondary": [], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "experiments/paper-ii-compute__Paper-II", "iso": "2026-09-04T22:58:34Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper II", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-002", "status": "open" }, { "id": "FALS-006", "status": "open" }, { "id": "FALS-007", "status": "retracted" }, { "id": "FALS-022", "status": "open" }, { "id": "FALS-031", "status": "open" }, { "id": "FALS-032", "status": "open" } ] } }, { "id": "T03", "group": "depth", "order": 3, "title": "The alignment exponent against the capability exponent", "index_name": "alignment exponent", "repository": "arc-principle-validation", "folder": "experiments/alignment-scaling__Papers-IV-a-b-c-d", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/alignment-scaling__Papers-IV-a-b-c-d", "label": "the alignment-scaling folder", "what": "the shared folder holding the runner and the instructions for all four Paper IV questions" }, "code": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/blob/main/experiments/alignment-scaling__Papers-IV-a-b-c-d/scripts/arc_alignment_scaling_v5.py", "label": "the version-five alignment-scaling runner", "what": "the runner behind the six subject-model finals in results/v5-final/" } }, "papers": [ { "slug": "paper-iii-alignment-scaling-problem", "label": "III" }, { "slug": "paper-iv-a-baked-in-vs-computed-alignment", "label": "IV.a" }, { "slug": "paper-iv-b-alignment-saturation-at-low-depth", "label": "IV.b" }, { "slug": "paper-iv-c-arc-align-benchmark", "label": "IV.c" }, { "slug": "paper-iv-d-the-effect-of-blinding-on-ai-alignment-evaluation", "label": "IV.d" } ], "decides": { "question": "Whether alignment scales with reasoning depth and how that compares with capability, measured across eight models under a blinded scoring protocol with bootstrap confidence intervals. The attempts register puts the same question this way: whether a stated value survives deletion of its statement, or was resting on a document.", "source": "the folder's REPRODUCE.md, and the attempts register for the same question in its own words" }, "run_command": "python3 scripts/arc_alignment_scaling_v5.py --model ", "run_note": "the command is taken once for each subject model, with a model flag; the folder's instructions file names the version-six runner as canonical, and the repository index lists that runner as not yet run", "requirements": "Python 3 with numpy, scipy, a hashing module and an HTTP client, and a key for each of the six subject models the repository index names for that version.", "run_prose": "The committed results are the version-five alignment-scaling runner's: the repository index records six subject-model finals under results/v5-final/ from that runner, and lists the version-six runner as not yet run. So the command that reproduces what is published is `python3 scripts/arc_alignment_scaling_v5.py` with a model flag, once for each of the six subject models the index names. That needs Python 3 with numpy, scipy, a hashing module and an HTTP client, and a key for each of those providers, and the exact library pins and the key setup are both left for the author to fill, so a fresh collection cannot be run from the instructions alone. The folder's own instructions file names the version-six runner as canonical and calls the earlier versions superseded. Start at the alignment-scaling folder.", "runnable_today": { "verdict": "no", "label": "not runnable for a fresh collection", "qualifier": "the recorded per-model results are committed, so the design and the arithmetic can be read", "missing": [ "keys for the six subject models the index names", "the exact library pins, marked for the author to fill", "the key setup, marked for the author to fill" ] }, "evidence_kind": "pilot measurement", "evidence_kind_basis": "A run collects fresh observations from eight live models under a protocol the repository itself grades a pilot.", "pass_means": "The instructions state what a faithful run brings back: the recorded exponents inside their bootstrap intervals, with the alignment exponent architecture dependent and distinct from the capability exponent.", "fail_means": "A run that lands outside those intervals is a finding against the recorded measurement.", "ceiling": "Neither outcome settles an ARC proposition, because the repository grades this folder a pilot, blind-scored at version five.", "reproducibility_note": "The design is checkable as it stands, because the models, the four token budgets of the depth sweep and the output schema are all stated. The benchmark paper states that its reference implementation stands at v5.4.2; the folder's instructions file names the version-six runner as canonical and calls the earlier versions superseded, and the repository index lists that version-six runner as not yet run.", "settles_prose": "The instructions state what a faithful run brings back: the recorded exponents inside their bootstrap intervals, with the alignment exponent architecture dependent and distinct from the capability exponent. A run that lands outside those intervals is a finding against the recorded measurement. Neither outcome settles an ARC proposition, because the repository grades this folder a pilot, blind-scored at version five. The design is checkable as it stands, because the models, the four token budgets of the depth sweep and the output schema are all stated. The benchmark paper states that its reference implementation stands at v5.4.2; the folder's instructions file names the version-six runner as canonical and calls the earlier versions superseded, and the repository index lists that version-six runner as not yet run.", "settles_merged": [], "protocol_status": "Pilot, blind-scored at version five; the published results are the version-five runner's and the version-six runner is listed as not yet run", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "REPRODUCE.md", "blob_sha": "7bf45875a534579df7984d3536cef0964db1f96c", "bytes": 3247, "secondary": [ { "file": "arc_alignment_scaling_v5.py", "blob_sha": "1b95c18d74b66e9c755f631ebf70c4158afb1cbb", "bytes": 457984, "role": "the runner behind the published version-five results, committed beside the superseded earlier versions and the unrun version-six runner" } ], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "experiments/alignment-scaling__Papers-IV-a-b-c-d", "iso": "2026-09-04T22:58:34Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Papers III, IV.a, IV.b, IV.c and IV.d, and Paper III does not link it", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-003", "status": "open" }, { "id": "FALS-009", "status": "open" }, { "id": "FALS-013", "status": "open" }, { "id": "FALS-014", "status": "open" }, { "id": "FALS-015", "status": "open" } ] } }, { "id": "T09", "group": "depth", "order": 4, "title": "The cross-programme re-analysis tools", "index_name": "cross-programme re-analysis tools", "repository": "arc-principle-validation", "folder": "experiments/analysis-tools__Cross-Programme", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/analysis-tools__Cross-Programme", "label": "the analysis-tools folder", "what": "the folder holding the re-analysis utilities and the instructions that run them" } }, "papers": [ { "slug": "paper-ii-experimental-validation", "label": "II" }, { "slug": "paper-iv-a-baked-in-vs-computed-alignment", "label": "IV.a" }, { "slug": "paper-iv-b-alignment-saturation-at-low-depth", "label": "IV.b" }, { "slug": "paper-iv-c-arc-align-benchmark", "label": "IV.c" }, { "slug": "paper-iv-d-the-effect-of-blinding-on-ai-alignment-evaluation", "label": "IV.d" }, { "slug": "paper-v-stewardship-gene", "label": "V" } ], "decides": { "question": "Nothing on its own. These are cross-programme utilities that aggregate and re-analyse the raw results the primary folders produced. The folder calls itself the checks the checkers layer and states that it generates no primary data.", "source": "the folder's REPRODUCE.md and its own description of itself" }, "run_command": "python3 analyze_alpha_align_v5.py ; python3 per_scorer_check.py", "run_note": null, "requirements": "Python 3 with numpy and scipy, and the primary results, which are committed. No API key.", "run_prose": "The instructions give `python3 analyze_alpha_align_v5.py` with a path to a results directory, and `python3 per_scorer_check.py`. Needs Python 3 with numpy and scipy, no key, and the primary results, which are committed; the version pins are left for the author to fill. Start at the analysis-tools folder.", "runnable_today": { "verdict": "yes", "label": "runnable today", "qualifier": "the version pins are left for the author to fill", "missing": [] }, "evidence_kind": "implementation check", "evidence_kind_basis": "It re-analyses results already written down and never touches a model, so it collects no observation of its own.", "pass_means": "A run tells a reader whether the published analysis follows from the results the primary folders wrote down.", "fail_means": "A run whose figures depart from the published analysis shows that the analysis does not follow from the recorded results.", "ceiling": "It cannot tell them whether those results are right, because this code never touches a model.", "reproducibility_note": "The repository grades this folder tooling, which it defines as code used by other experiments and producing no result of its own.", "settles_prose": "A run tells a reader whether the published analysis follows from the results the primary folders wrote down. It cannot tell them whether those results are right, because this code never touches a model. The repository grades this folder tooling, which it defines as code used by other experiments and producing no result of its own.", "settles_merged": [ "fail_means" ], "protocol_status": "Tooling: code used by other experiments, producing no result of its own", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "REPRODUCE.md", "blob_sha": "024708ac66ca429fc59e07a93a7a27fe2d53a485", "bytes": 1468, "secondary": [], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "experiments/analysis-tools__Cross-Programme", "iso": "2026-07-05T09:20:43Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Papers II, IV.a, IV.b, IV.c, IV.d and V, and none of the six links this test", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [] } }, { "id": "T15", "group": "depth", "order": 5, "title": "The ARC scaling challenge, legacy toolkit", "index_name": "legacy toolkit", "repository": "arc-scaling-challenge", "folder": "(repository root: analysis/, tools/, tests/, protocols/, examples/, figures/, submissions/)", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-scaling-challenge/blob/main/README.md", "label": "the arc-scaling-challenge README", "what": "the repository README, which is where the run instructions are" } }, "papers": [ { "slug": "paper-i-arc-principle", "label": "I" }, { "slug": "paper-ii-experimental-validation", "label": "II" }, { "slug": "paper-iii-alignment-scaling-problem", "label": "III" } ], "decides": { "question": "The questions in that toolkit version. Its README offers standardised tools to measure the exponent in your own systems, compare results across domains and falsify the framework if it fails, with five predictions and ten falsification criteria.", "source": "the repository README" }, "run_command": "git clone; pip install -r requirements.txt (or make install); python analysis/python/arc_analysis.py --data examples/data/example_ai_sequential_parallel.csv ; make verify ; python tools/beta_estimator.py --alpha 2.2 ; python -m pytest tests/ -v. R path: source('analysis/r/install_packages.R') then source('analysis/r/arc_analysis.R').", "run_note": null, "requirements": "numpy>=1.20.0, scipy>=1.7.0, pandas>=1.3.0, matplotlib>=3.4.0, seaborn>=0.11.0, pytest>=6.0.0, pytest-cov>=2.12.0. Domain protocols require a minimum of five data points across recursive depth and both sequential and parallel conditions where applicable. No API key.", "run_prose": "Clone the repository, `pip install -r requirements.txt`, then `python analysis/python/arc_analysis.py` against the committed example data, `make verify`, and `python -m pytest tests/ -v`. An R path is given too. Needs the pinned Python stack and no key. Start at the arc-scaling-challenge README, which is where the instructions are.", "runnable_today": { "verdict": "yes", "label": "runnable today", "qualifier": "install, run, verify and expected output are all given and the example data is committed", "missing": [] }, "evidence_kind": "pilot measurement", "evidence_kind_basis": "A reader running it on their own pipeline collects fresh observations, and the estate's status line requires reconciliation with the current theory protocol before a deciding study.", "pass_means": "A run that recovers the exponent the toolkit was written to recover, on the data supplied to it, shows the implementation works.", "fail_means": "A run that does not bears on that toolkit version and on nothing later.", "ceiling": "Example recovery or a toolkit PASS is not confirmation of the current P1-P22 propositions.", "reproducibility_note": "Its stated expected output is a sequential exponent of about 2.2 and its results table records that estimate, while the other repository records the same estimate as retracted and replaced by about 0.49, which is sub-linear.", "settles_prose": "A run that recovers the exponent the toolkit was written to recover, on the data supplied to it, shows the implementation works. A run that does not bears on that toolkit version and on nothing later. Its repository README retains earlier numerical claims and proposition numbering. Example recovery or a toolkit PASS is not confirmation of the current P1-P22 propositions. Its stated expected output is a sequential exponent of about 2.2 and its results table records that estimate, while the other repository records the same estimate as retracted and replaced by about 0.49, which is sub-linear.", "settles_merged": [], "protocol_status": "Legacy toolkit; reconcile with the current theory protocol before a deciding study", "caveat": "Its repository README retains earlier numerical claims and proposition numbering. Example recovery or a toolkit PASS is not confirmation of the current P1-P22 propositions.", "licence_key": "mit", "instructions": { "present": true, "file": "README.md", "blob_sha": "992b32b6cd03f562ae09130f8b3fc234f866ffb9", "bytes": 10188, "secondary": [], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "README.md", "iso": "2026-07-02T02:55:05Z", "is_the_test_folder": false, "print": true, "label": "its README" }, { "path": "analysis", "iso": "2026-02-09T23:59:03Z", "is_the_test_folder": true, "print": true, "label": "its analysis, tools, tests, protocols and examples directories", "join": ", and ", "covers": [ "analysis", "tools", "tests", "protocols", "examples" ] }, { "path": "tools", "iso": "2026-02-09T23:59:03Z", "is_the_test_folder": true, "print": false }, { "path": "tests", "iso": "2026-02-09T23:59:03Z", "is_the_test_folder": true, "print": false }, { "path": "protocols", "iso": "2026-02-09T23:41:46Z", "is_the_test_folder": true, "print": false }, { "path": "examples", "iso": "2026-02-09T23:59:03Z", "is_the_test_folder": true, "print": false } ], "papers_clause": "Papers I, II and III, and none of the three links it", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-002", "status": "open" }, { "id": "FALS-006", "status": "open" }, { "id": "FALS-007", "status": "retracted" }, { "id": "FALS-009", "status": "open-formal-consistency" } ] } }, { "id": "T04", "group": "intervention", "order": 1, "title": "The stewardship intervention across depth", "index_name": "stewardship intervention", "repository": "arc-principle-validation", "folder": "experiments/eden-intervention__Paper-V", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/eden-intervention__Paper-V", "label": "the eden-intervention folder", "what": "the folder holding the runner, the recorded results and the instructions" } }, "papers": [ { "slug": "paper-v-stewardship-gene", "label": "Paper V" } ], "decides": { "question": "Whether the Eden Protocol intervention holds as capability scales, applied across reasoning depth on six models at four depth levels with ten prompts each.", "source": "the folder's REPRODUCE.md" }, "run_command": "eden_protocol_scaling_test_v3.py", "run_note": "the runner is named without its scripts/ prefix, and is taken once per model with the flags its own argument parser lists", "requirements": "Python 3 with numpy, scipy and a model client, and a key for each of the six providers.", "run_prose": "The runner is `eden_protocol_scaling_test_v3.py`, taken once per model with the flags its own argument parser lists, and like the others it is named without its `scripts/` prefix. A key for each of the six providers is required, on top of Python 3 with numpy, scipy and a model client, and neither those keys nor the library versions are pinned down anywhere in the folder. Start at the eden-intervention folder.", "runnable_today": { "verdict": "no", "label": "not runnable for a fresh collection", "qualifier": "the recorded per-model results are committed, so the design and the arithmetic can be read", "missing": [ "keys for the six providers", "library versions, pinned down nowhere in the folder" ] }, "evidence_kind": "pilot measurement", "evidence_kind_basis": "A run collects fresh observations from six live models under a protocol the repository grades single-scorer and not blinded.", "pass_means": "The instructions state the expected result: the intervention effect against depth reproduced within run variance across the four depth levels, the six models and the ten prompts.", "fail_means": "A run outside that variance is a finding against what this folder records.", "ceiling": "Neither outcome settles an ARC proposition, because the repository grades this folder a pilot, single-scorer and not blinded.", "reproducibility_note": "The models, the depth levels and the prompt count are stated, so the design is checkable as it stands. The repository records the combined figure of 6.3 x 10 to the minus 21 as withdrawn, and five per-model results as standing.", "settles_prose": "The instructions state the expected result: the intervention effect against depth reproduced within run variance across the four depth levels, the six models and the ten prompts. A run outside that variance is a finding against what this folder records. Neither outcome settles an ARC proposition, because the repository grades this folder a pilot, single-scorer and not blinded. The models, the depth levels and the prompt count are stated, so the design is checkable as it stands. The repository records the combined figure of 6.3 x 10 to the minus 21 as withdrawn, and five per-model results as standing.", "settles_merged": [], "protocol_status": "Pilot, single-scorer and not blinded", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "REPRODUCE.md", "blob_sha": "11b30b8e3a8d398bd2898c4c0352578754a848e3", "bytes": 1625, "secondary": [], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "experiments/eden-intervention__Paper-V", "iso": "2026-07-05T09:20:43Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper V, whose page links no folder at all, only the two repository roots", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-005", "status": "open" } ] } }, { "id": "T05", "group": "intervention", "order": 2, "title": "The self-modifying network", "index_name": "self-modifying network", "repository": "arc-principle-validation", "folder": "experiments/honey-architecture__Paper-VI", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/honey-architecture__Paper-VI", "label": "the honey-architecture folder", "what": "the folder holding the scripts, the recorded results and the instructions" } }, "papers": [ { "slug": "paper-vi-honey-architecture", "label": "Paper VI" } ], "decides": { "question": "Whether a network that genuinely modifies its own hyperparameters during training holds up on capability, safety, fairness, adversarial robustness and complexity scaling.", "source": "the folder's REPRODUCE.md and the paper page's list of committed scripts" }, "run_command": "python3 scripts/merge_honey_test_results.py", "run_note": "that command aggregates the recorded results; the instructions name no primary generator and leave its command for the author to fill", "requirements": "Python 3 with numpy and matplotlib.", "run_prose": "The instructions name no primary generator, leave its command for the author to fill and give only an example form; the one real command they give is `python3 scripts/merge_honey_test_results.py`, which aggregates the recorded results. The paper page names six committed scripts the folder does hold. Needs Python 3 with numpy and matplotlib. Start at the honey-architecture folder.", "runnable_today": { "verdict": "no", "label": "not runnable from its instructions", "qualifier": "the folder's scripts directory holds the generator and its three successors, with the seeds and cycles recorded, so a run is reconstructable by reading the scripts", "missing": [ "the primary generator's command, left for the author to fill" ] }, "evidence_kind": "pilot measurement", "evidence_kind_basis": "A run collects fresh observations under a protocol the repository grades mechanistic in simulation and exploratory on live models.", "pass_means": "The instructions state the expected output: result files matching the recorded summary fields for the capability, safety and drag arms, and figures matching the ones in the folder.", "fail_means": "A run that does not match is a finding against those artefacts.", "ceiling": "Neither outcome settles an ARC proposition, because the repository grades this folder mechanistic in simulation and exploratory on live models, and it defines mechanistic as a mechanism shown in a toy system with no claim about any deployed system.", "reproducibility_note": "The instructions also contradict the folder: they declare the generator uncommitted while the folder's scripts directory holds it and its three successors, and the seeds and cycles for each version are recorded, so a run is reconstructable by reading the scripts and not from the instructions alone.", "settles_prose": "The instructions state the expected output: result files matching the recorded summary fields for the capability, safety and drag arms, and figures matching the ones in the folder. A run that does not match is a finding against those artefacts. Neither outcome settles an ARC proposition, because the repository grades this folder mechanistic in simulation and exploratory on live models, and it defines mechanistic as a mechanism shown in a toy system with no claim about any deployed system. The instructions also contradict the folder: they declare the generator uncommitted while the folder's scripts directory holds it and its three successors, and the seeds and cycles for each version are recorded, so a run is reconstructable by reading the scripts and not from the instructions alone.", "settles_merged": [], "protocol_status": "Mechanistic in simulation, exploratory on live models", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "REPRODUCE.md", "blob_sha": "9ec7ef9a4dfc9a11e7be32b7999148b7a37a4328", "bytes": 3269, "secondary": [], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "experiments/honey-architecture__Paper-VI", "iso": "2026-09-04T22:58:34Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper VI", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-008", "status": "open" }, { "id": "FALS-016", "status": "open" } ] } }, { "id": "T12", "group": "intervention", "order": 3, "title": "The load-bearing proof harness", "index_name": "load-bearing proof harness", "repository": "arc-principle-validation", "folder": "papers/Paper-VIII-The-Load-Bearing-Proof/experiments", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/papers/Paper-VIII-The-Load-Bearing-Proof/experiments", "label": "the Paper VIII experiments folder", "what": "the folder holding the harness, its tests and the instructions that run them" } }, "papers": [ { "slug": "paper-viii-the-load-bearing-proof", "label": "Paper VIII" } ], "decides": { "question": "Whether intrinsic self-modification produces increasing marginal gains, and whether verification drag is load-bearing for stability. There are four conditions: static, one that accepts a self-modification on capability gain alone, one that accepts only when capability, safety and the entangled objective all improve, and a drag control.", "source": "the folder's README.md and its REPRODUCE.md" }, "run_command": "python3 src/run_train.py ; python3 -m pytest", "run_note": "taken from the architectural harness directory", "requirements": "Python 3.11 or later with torch, numpy, scipy, pyyaml, matplotlib and pytest, and no key for the simulation. The behavioural runners need your own model keys, and the weight experiment needs a fine-tuning stack and a base model.", "run_prose": "From the architectural harness directory, `python3 src/run_train.py` with a config, and a pytest line for its tests. Needs Python 3.11 or later with torch, numpy, scipy, pyyaml, matplotlib and pytest, and no key for the simulation; the behavioural runners need your own model keys, and the weight experiment needs a fine-tuning stack and a base model. Of every instructions file this register records, it is the most complete: it names the two places where the deposit's layout differs from the layout the code expects, and warns that in both the code will keep going and do the wrong thing rather than stop. Start at the Paper VIII experiments folder.", "runnable_today": { "verdict": "yes", "label": "runnable today", "qualifier": "the simulation needs no key; the behavioural and weight arms need keys and a fine-tuning stack of your own", "missing": [] }, "evidence_kind": "pilot measurement", "evidence_kind_basis": "The behavioural arms collect fresh observations from live models, and the repository grades the folder unblinded and not replicated.", "pass_means": "A pass shows the harness does what its own README says.", "fail_means": "A failure shows the deposit and the code disagree.", "ceiling": "The instructions state it plainly: rerunning tests whether the procedure behaves as described, and does not replicate the finding, because a replication needs a design fixed in advance, a scorer who cannot see the condition and an analysis stated before the outcomes.", "reproducibility_note": "That study is written and dated as a draft registration awaiting human filing. The repository grades this folder a pilot, unblinded and not replicated, and records weight-level structural entanglement as inconclusive at the training scale tested.", "settles_prose": "The instructions state it plainly: rerunning tests whether the procedure behaves as described, and does not replicate the finding, because a replication needs a design fixed in advance, a scorer who cannot see the condition and an analysis stated before the outcomes. So a pass shows the harness does what its own README says, a failure shows the deposit and the code disagree, and neither is a replication. That study is written and dated as a draft registration awaiting human filing. The repository grades this folder a pilot, unblinded and not replicated, and records weight-level structural entanglement as inconclusive at the training scale tested.", "settles_merged": [ "pass_means", "fail_means" ], "protocol_status": "Pilot, unblinded and not replicated", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "README.md", "blob_sha": "1132a297855baedfcc8a58c360e207a45121c9cb", "bytes": 15104, "secondary": [ { "file": "REPRODUCE.md", "blob_sha": "f6b2b7305b0aea53e265c32f9ece1eb59b7007ac", "bytes": 4664, "role": "the reproduce file beside the README, naming the two places where the deposit's layout differs from the layout the code expects" } ], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "papers/Paper-VIII-The-Load-Bearing-Proof/experiments", "iso": "2026-09-07T00:33:21Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper VIII, whose page links the repository roots and not this folder", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-017", "status": "partial" } ] } }, { "id": "T16", "group": "correction", "order": 1, "title": "The falsification challenge, openly licensed", "index_name": "falsification challenge", "repository": "arc-scaling-challenge", "folder": "coupled-coscaling", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-scaling-challenge/tree/main/coupled-coscaling", "label": "the coupled-coscaling folder", "what": "the folder holding the harness, its tests and the README that runs them" } }, "papers": [ { "slug": "paper-x-coupled-coscaling-correction", "label": "Paper X" } ], "decides": { "question": "Whether the harness can be made to print FAIL. It is a self-contained challenge to the claim that the stability of recursive self-improvement is governed by a single exponent inequality, and that a hard takeoff is alignment-stable exactly when that inequality holds, with the speed of the explosion not changing the verdict.", "source": "the folder's README.md" }, "run_command": "pip install numpy scipy matplotlib pytest; python experiment_coscaling.py; pytest test_coscaling.py -q", "run_note": null, "requirements": "The pinned Python stack in the README. No API key.", "run_prose": "`pip install numpy scipy matplotlib pytest`, then `python experiment_coscaling.py`, which runs ten experiments and prints PASS or FAIL for each, then `pytest test_coscaling.py -q` for twelve internal-consistency assertions. No key. Start at the coupled-coscaling folder.", "runnable_today": { "verdict": "yes", "label": "runnable today", "qualifier": "install, run and expected output are all given and the harness prints its own verdict", "missing": [] }, "evidence_kind": "implementation check", "evidence_kind_basis": "The deductive checks integrate the model's own differential equation, so a run verifies the derivation and the solver and collects no observation.", "pass_means": "The harness exits zero only when every internal-consistency check matches its closed-form prediction.", "fail_means": "A FAIL is the outcome the challenge invites, and the README lists the five kill conditions that would refute the thesis and the one that would downgrade only the claim that the mechanism is like quantum error correction.", "ceiling": "Neither outcome tests the model against a real system: the README states that the deductive checks integrate the model's own differential equation, so they verify the derivation and the solver, and it says the genuine empirical falsifier is the open problem.", "reproducibility_note": "The reference run is recorded as ten of ten passing with no kill condition triggered.", "settles_prose": "The harness exits zero only when every internal-consistency check matches its closed-form prediction, so a FAIL is the outcome the challenge invites, and the README lists the five kill conditions that would refute the thesis and the one that would downgrade only the claim that the mechanism is like quantum error correction. The reference run is recorded as ten of ten passing with no kill condition triggered. Neither outcome tests the model against a real system: the README states that the deductive checks integrate the model's own differential equation, so they verify the derivation and the solver, and it says the genuine empirical falsifier is the open problem.", "settles_merged": [ "pass_means", "fail_means" ], "protocol_status": "Reference run recorded; the genuine empirical falsifier is the open problem", "caveat": null, "licence_key": "mit", "instructions": { "present": true, "file": "README.md", "blob_sha": "5d01aa95ac4cb329cea6bcd7c438c6b2a4388da2", "bytes": 6145, "secondary": [], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "coupled-coscaling", "iso": "2026-06-27T08:57:19Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper X, whose page links the scaling-challenge repository root and not this folder", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-021", "status": "open" }, { "id": "FALS-039", "status": "open" }, { "id": "FALS-040", "status": "open" }, { "id": "FALS-041", "status": "open" } ] } }, { "id": "T13", "group": "correction", "order": 2, "title": "The theorem checks in the paper folder", "index_name": "theorem checks", "repository": "arc-principle-validation", "folder": "papers/Paper-X-Coupled-CoScaling-Correction/code", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/papers/Paper-X-Coupled-CoScaling-Correction", "label": "the Paper X paper folder", "what": "the paper folder, which is where the instructions, the Makefile and the manifest are" }, "code": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/papers/Paper-X-Coupled-CoScaling-Correction/code", "label": "the code folder", "what": "the code directory the repository index points at, which carries no instructions of its own" } }, "papers": [ { "slug": "paper-x-coupled-coscaling-correction", "label": "Paper X" } ], "decides": { "question": "The same inequality as the challenge copy, by ten experiments printing PASS or FAIL against eight predictions, with a separate script that verifies the theorems independently.", "source": "the paper folder's README.md and its REPRODUCIBILITY.md" }, "run_command": "python experiment_coscaling.py; pytest test_coscaling.py -q", "run_note": "the paper folder carries a Makefile, a Dockerfile that builds an offline image running only the deterministic checks, a requirements file and a hashed manifest", "requirements": "The pinned requirements in the paper folder, or the Dockerfile image. No API key.", "run_prose": "The paper folder carries a Makefile, a Dockerfile that builds an offline image running only the deterministic checks, a requirements file and a hashed manifest. Its commands are the challenge copy's, `python experiment_coscaling.py` then `pytest test_coscaling.py -q`, and no key is needed. Start at the Paper X paper folder, because the code folder the repository index points at carries no instructions of its own.", "runnable_today": { "verdict": "yes", "label": "runnable today", "qualifier": "the paper folder carries the Makefile, the Dockerfile, the requirements file and the hashed manifest", "missing": [] }, "evidence_kind": "implementation check", "evidence_kind_basis": "A PASS verifies the code against the mathematics on deterministic inputs and collects no observation.", "pass_means": "A PASS verifies that the code matches the mathematics.", "fail_means": "The runbook beside it names a script in the code folder as the independent verification of the theorems, so a failure there would be a finding about the derivation rather than about the world.", "ceiling": "Neither outcome tests the law against any real system, and the README says that empirical test is the genuine falsifier and remains the open problem.", "reproducibility_note": null, "settles_prose": "A PASS verifies that the code matches the mathematics. It does not test the law against any real system, and the README says that empirical test is the genuine falsifier and remains the open problem. The runbook beside it names a script in the code folder as the independent verification of the theorems, so a failure there would be a finding about the derivation rather than about the world.", "settles_merged": [ "ceiling" ], "protocol_status": "Deterministic checks in the paper folder; the empirical test remains the open problem", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "README.md", "blob_sha": "b1b79cb25fe4a493d372bc638d1650da16282389", "bytes": 12116, "secondary": [ { "file": "REPRODUCIBILITY.md", "blob_sha": "cdac7adf4cf26ce60cbbb28d74a4bd4c489e18ea", "bytes": 4073, "role": "the reproducibility record beside the README in the paper folder" } ], "missing": [], "tail": ", both in the paper folder" }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "papers/Paper-X-Coupled-CoScaling-Correction/code", "iso": "2026-06-27T12:56:04Z", "is_the_test_folder": true, "print": true, "label": "code folder" }, { "path": "papers/Paper-X-Coupled-CoScaling-Correction", "iso": "2026-08-22T13:31:14Z", "is_the_test_folder": false, "print": true, "label": "the paper folder around it", "join": ", " } ], "papers_clause": "Paper X", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-021", "status": "open" }, { "id": "FALS-039", "status": "open" }, { "id": "FALS-040", "status": "open" }, { "id": "FALS-041", "status": "open" } ] } }, { "id": "T14", "group": "correction", "order": 3, "title": "The real-model loop", "index_name": "real-model loop", "repository": "arc-principle-validation", "folder": "papers/Paper-X-Coupled-CoScaling-Correction/experiments", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/papers/Paper-X-Coupled-CoScaling-Correction/experiments", "label": "the Paper X experiments folder", "what": "the folder holding the runbook, the protocols and the harness" } }, "papers": [ { "slug": "paper-x-coupled-coscaling-correction", "label": "Paper X" } ], "decides": { "question": "Whether the mechanism shows up on a real model. A model repeatedly improves code under three conditions: pure visible-score pressure with no integrity audit, every round followed by an integrity corrector, and an extra edit pass with no audit, to control for the extra compute. Capability is the hidden-test pass rate and misalignment is a fused score.", "source": "the folder's RUN_REAL_MODEL_EXPERIMENT.md" }, "run_command": "python3 realmodel_coscaling_v2.py --selftest --rounds 3 --seeds 1 --tasks arith --conditions coupled decoupled --speeds steady", "run_note": "run from experiments/scripts, after changing into the paper folder; the self-test is Step A and three further steps climb to a credible run of 540 trajectories", "requirements": "Python 3.11, with 3.10 or later working, the pinned requirements, two provider keys for the intended run, and a disposable network-isolated sandbox, because the harness executes model-generated code to grade capability.", "run_prose": "The runbook opens by stating that a reader can land there with no prior context and run it correctly. Step A is a self-test that needs no key and about five seconds: from the paper folder, change into `experiments/scripts` and run `python3 realmodel_coscaling_v2.py --selftest --rounds 3 --seeds 1 --tasks arith --conditions coupled decoupled --speeds steady`. Three further steps climb to a credible run of 540 trajectories. Needs Python 3.11, with 3.10 or later working, the pinned requirements, two provider keys for the intended run, and a disposable network-isolated sandbox, because the harness executes model-generated code to grade capability. Start at the Paper X experiments folder.", "runnable_today": { "verdict": "yes", "label": "runnable today", "qualifier": "the self-test needs no key and about five seconds; the intended run needs two provider keys", "missing": [] }, "evidence_kind": "pilot measurement", "evidence_kind_basis": "The intended run collects fresh trajectories from live models at pilot scale, under a protocol the runbook itself calls a pilot.", "pass_means": "A positive result is mechanism-level evidence, and a single cheap-engine pilot earns a first real-model signal consistent with the coupling mechanism on one model at pilot scale and nothing larger.", "fail_means": "A null or a not-identifiable result is reported as such rather than buried, and capability saturating is an honest null that says to pick a model with more headroom.", "ceiling": "The runbook states the boundary itself: a positive result is not proof of alignment and not proof of the inequality in general, and it reaches one model at pilot scale and nothing larger.", "reproducibility_note": "Its own reading rule is that the self-test output is not data, and that the self-test flag must be false for anything a reader intends to cite.", "settles_prose": "The runbook states the boundary itself: a positive result is mechanism-level evidence, not proof of alignment and not proof of the inequality in general, and a single cheap-engine pilot earns a first real-model signal consistent with the coupling mechanism on one model at pilot scale and nothing larger. A null or a not-identifiable result is reported as such rather than buried, and capability saturating is an honest null that says to pick a model with more headroom. Its own reading rule is that the self-test output is not data, and that the self-test flag must be false for anything a reader intends to cite.", "settles_merged": [ "pass_means", "ceiling" ], "protocol_status": "Pilot at cheap-engine scale; mechanism-level evidence only", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "RUN_REAL_MODEL_EXPERIMENT.md", "blob_sha": "622024dc30a4937c06aedd821db055a35013c1c6", "bytes": 12106, "secondary": [ { "file": "PROTOCOL_V2.md", "blob_sha": "e3018fc15f7722ac2ed20faba5794208647b1545", "bytes": 8042, "role": "the concise protocol the runbook names, hypotheses H1 to H4 and the result taxonomy" }, { "file": "CONFIRMATORY_PROTOCOL_V2.md", "blob_sha": "2ab61b4a330ee5e285460423e66bc06282600f4d", "bytes": 10155, "role": "the detailed protocol the runbook names, extra statistics and adversarial detail" }, { "file": "README_V2_UPGRADE.md", "blob_sha": "04a5d5fcf8863f7a7812be3dcd524537b1ffcb0a", "bytes": 3618, "role": "the orientation between the first and second versions" }, { "file": "PROTOCOL.md", "blob_sha": "7af3f4f5ec2c260bbb009c48e2b95024f2f99506", "bytes": 17597, "role": "an earlier protocol still in the folder, which the runbook does not name" } ], "missing": [], "clause": "The runbook is RUN_REAL_MODEL_EXPERIMENT.md 622024d, and it sends a reader to the second version of the protocol: PROTOCOL_V2.md e3018fc for the hypotheses and the result taxonomy, CONFIRMATORY_PROTOCOL_V2.md 2ab61b4 for the detail, and README_V2_UPGRADE.md 04a5d5f between the two versions. The earlier PROTOCOL.md 7af3f4f is still in the folder and the runbook does not name it." }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "papers/Paper-X-Coupled-CoScaling-Correction/experiments", "iso": "2026-07-03T05:52:41Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper X, whose page links the paper folder above this one and not this folder", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-021", "status": "open" }, { "id": "FALS-039", "status": "open" }, { "id": "FALS-040", "status": "open" }, { "id": "FALS-041", "status": "open" }, { "id": "FALS-044", "status": "open" } ] } }, { "id": "T17", "group": "correction", "order": 4, "title": "The real-model loop, openly licensed copy", "index_name": "real-model loop, openly licensed", "repository": "arc-scaling-challenge", "folder": "coupled-coscaling/realmodel", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-scaling-challenge/tree/main/coupled-coscaling/realmodel", "label": "the realmodel folder", "what": "the folder holding the protocol file, the scripts and the recorded results" } }, "papers": [ { "slug": "paper-x-coupled-coscaling-correction", "label": "Paper X" } ], "decides": { "question": "The same question as the copy above, put by the parent README as the genuine falsifier that tests the model against a real system, with capability scored by executing candidates against hidden tests and misalignment scored blind.", "source": "the parent folder's README.md and this folder's PROTOCOL.md" }, "run_command": "scripts/realmodel_coscaling.py", "run_note": "the harness is named by the parent README; there is no command line in this folder", "requirements": "The six-provider setup the parent README names.", "run_prose": "There is no command line in this folder and no README either. The parent README names `scripts/realmodel_coscaling.py` as the harness and points to the committed protocol file for the predictions and the wiring, and the near-identical harness with the full copy-paste ladder is the entry above, under a licence that does not permit re-use. Needs the six-provider setup the parent README names. Start at the realmodel folder.", "runnable_today": { "verdict": "partly", "label": "runnable in part", "qualifier": "the parent README names the harness and the committed protocol file carries the predictions and the wiring", "missing": [ "a command line of its own", "a README in the folder" ] }, "evidence_kind": "pilot measurement", "evidence_kind_basis": "A run collects fresh observations from live models under a protocol that is not frozen, blinded and pre-analysed.", "pass_means": "A run showing drift under the decoupled arm and a corrector that bounds it supports the mechanism on that model and nothing wider.", "fail_means": "The parent README states the clean refutation: either a decoupled model whose misalignment fraction stays at the floor as capability rises, so there was no drift to correct, or a coupled corrector that cannot bring a reward hack down.", "ceiling": "Neither outcome reaches beyond that model: a run supports the mechanism on that model and nothing wider.", "reproducibility_note": "The first real arm is recorded as a null contrast rather than a refutation: from a seeded reward hack both arms removed the hack at round one and stayed general, so on that task the frontier model does not drift. The recorded corrector result is a single-model probe on a frozen hack, and the README names the other five models as the open target.", "settles_prose": "The parent README states the clean refutation: either a decoupled model whose misalignment fraction stays at the floor as capability rises, so there was no drift to correct, or a coupled corrector that cannot bring a reward hack down. A run showing drift under the decoupled arm and a corrector that bounds it supports the mechanism on that model and nothing wider. The first real arm is recorded as a null contrast rather than a refutation: from a seeded reward hack both arms removed the hack at round one and stayed general, so on that task the frontier model does not drift. The recorded corrector result is a single-model probe on a frozen hack, and the README names the other five models as the open target.", "settles_merged": [ "ceiling" ], "protocol_status": "Single-model probe on a frozen hack; the other five models are the open target", "caveat": null, "licence_key": "mit", "instructions": { "present": true, "file": "PROTOCOL.md", "blob_sha": "16cc8d663a956b42055bce79693e47020ee6a7e5", "bytes": 12854, "secondary": [], "missing": [], "tail": ", and no README in the folder" }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "coupled-coscaling/realmodel", "iso": "2026-06-27T08:57:19Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper X, whose page links the challenge repository root and no folder inside it", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-021", "status": "open" }, { "id": "FALS-044", "status": "open" } ] } }, { "id": "T06", "group": "form", "order": 1, "title": "The fifty-domain validation", "index_name": "fifty-domain validation", "repository": "arc-principle-validation", "folder": "experiments/cauchy-unification__Paper-VII", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/cauchy-unification__Paper-VII", "label": "the cauchy-unification folder", "what": "the folder holding the harness, the manifest and the instructions" } }, "papers": [ { "slug": "paper-vii-cauchy-unification", "label": "Paper VII" } ], "decides": { "question": "Whether the form of a scaling law is constrained by the composition operator of the underlying recursive process. The primary script is a tiered fifty-domain harness with manifest-backed inputs, a strict empirical endpoint and explicit evidence tiers.", "source": "the folder's README.md" }, "run_command": "python3 scripts/arc_50_domain_universal_test.py", "run_note": "the instructions list eight further scripts beside the canonical command", "requirements": "Python 3 with NumPy and SciPy. No API key; every input is embedded in the manifest with its provenance citation and the output is deterministic.", "run_prose": "The instructions give `python3 scripts/arc_50_domain_universal_test.py` as the canonical command, and list eight further scripts beside it. Needs Python 3 with NumPy and SciPy, and no key; every input is embedded in the manifest with its provenance citation and the output is deterministic. Start at the cauchy-unification folder.", "runnable_today": { "verdict": "yes", "label": "runnable today", "qualifier": "the canonical command, the inputs and the expected outputs are all given", "missing": [] }, "evidence_kind": "implementation check", "evidence_kind_basis": "A run regenerates recorded outputs from manifest-backed inputs and collects no fresh observation.", "pass_means": "A run that returns the family-match counts and the probabilities the folder records, tier by tier, shows the harness reproduces what is recorded.", "fail_means": "A run that returns different ones is a finding about those recorded outputs.", "ceiling": "Neither outcome settles what the paper drew from them, because the paper is under correction: the repository states that the functional-equation grid has four cells rather than three, and that the primary statistic is a permutation test conditioned on both marginals rather than the binomial reported at the time, so the recorded outputs stand and the reading placed on them does not.", "reproducibility_note": "The folder grades its primary script supporting. Its twenty-domain script is labelled legacy with permissive scoring and is not the canonical headline, its saved null control is a lightweight first pass rather than a converged null surface, and one R script is documented as blocked because R is not installed locally.", "settles_prose": "A run that returns the family-match counts and the probabilities the folder records, tier by tier, shows the harness reproduces what is recorded; a run that returns different ones is a finding about those recorded outputs. Neither outcome settles what the paper drew from them, because the paper is under correction: the repository states that the functional-equation grid has four cells rather than three, and that the primary statistic is a permutation test conditioned on both marginals rather than the binomial reported at the time, so the recorded outputs stand and the reading placed on them does not. The folder grades its primary script supporting. Its twenty-domain script is labelled legacy with permissive scoring and is not the canonical headline, its saved null control is a lightweight first pass rather than a converged null surface, and one R script is documented as blocked because R is not installed locally.", "settles_merged": [ "pass_means", "fail_means" ], "protocol_status": "Supporting, and the paper is under correction", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "README.md", "blob_sha": "74c5265023ecb247a562551ac62c88f5b989f77f", "bytes": 9327, "secondary": [], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "experiments/cauchy-unification__Paper-VII", "iso": "2026-09-04T22:58:34Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper VII", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-004", "status": "partial" } ] } }, { "id": "T07", "group": "form", "order": 2, "title": "The dimensional relation scripts", "index_name": "dimensional relation scripts", "repository": "arc-principle-validation", "folder": "experiments/domain-validation__Foundational-and-Origin", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/domain-validation__Foundational-and-Origin", "label": "the domain-validation folder", "what": "the folder the paper sends readers to, holding the scripts, the results and the evidence pack" } }, "papers": [ { "slug": "foundational", "label": "Foundational" }, { "slug": "on-the-origin-of-scaling-laws", "label": "On the Origin of Scaling Laws" } ], "decides": { "question": "The dimensional relation across physics and other domains. On the Origin of Scaling Laws links this folder as its validation code and domain datasets.", "source": "On the Origin of Scaling Laws, which links this folder as its validation code and domain datasets" }, "run_command": "", "run_note": "the folder holds no run command, no dependency list, no entry point and no expected output", "requirements": "Not stated anywhere in the folder.", "run_prose": "There is no way to run it as published, and what is missing is the instructions file. The folder holds fourteen Python scripts, thirteen recorded result files, and an evidence pack of four documents among them a claim evidence ledger and a memo to the paper writer. It holds no run command, no dependency list, no entry point and no expected output. Start at the domain-validation folder, which is the folder the paper sends readers to.", "runnable_today": { "verdict": "no", "label": "not runnable", "qualifier": null, "missing": [ "an instructions file", "a run command", "a dependency list", "an entry point", "an expected output" ] }, "evidence_kind": "implementation check", "evidence_kind_basis": "The repository grades the folder supporting and mathematical, which it defines as a computational validation of a step rather than an empirical result. This grades what a run would be, because the folder cannot be run as published.", "pass_means": "A run settles nothing until an instructions file lands.", "fail_means": "Everything in the folder can be read; none of it can be produced again.", "ceiling": "The repository grades this folder supporting and mathematical, which it defines as a computational validation of a step rather than an empirical result, and it records that one file in the evidence pack is cited from outside the repository and does not move.", "reproducibility_note": "The Paper VII folder states that these scripts were developed here and collected there, and several filenames appear in both.", "settles_prose": "Nothing, until an instructions file lands. Everything in the folder can be read; none of it can be produced again. The repository grades this folder supporting and mathematical, which it defines as a computational validation of a step rather than an empirical result, and it records that one file in the evidence pack is cited from outside the repository and does not move. The Paper VII folder states that these scripts were developed here and collected there, and several filenames appear in both.", "settles_merged": [ "pass_means" ], "protocol_status": "Supporting and mathematical", "caveat": null, "licence_key": "reserved", "instructions": { "present": false, "file": null, "blob_sha": null, "bytes": null, "secondary": [], "missing": [ "an instructions file", "a run command", "a dependency list", "an expected output" ] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "experiments/domain-validation__Foundational-and-Origin", "iso": "2026-09-04T22:58:34Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Foundational, whose page does not link it, and On the Origin of Scaling Laws", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-018", "status": "open" }, { "id": "FALS-020", "status": "untested" }, { "id": "FALS-034", "status": "untested" } ] } }, { "id": "T08", "group": "decidability", "order": 1, "title": "The blind prediction test", "index_name": "blind prediction test", "repository": "arc-principle-validation", "folder": "experiments/blind-prediction-test__Paper-III-and-Foundational", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/blind-prediction-test__Paper-III-and-Foundational", "label": "the blind-prediction-test folder", "what": "the folder holding the script, the forensic analysis and the results file" } }, "papers": [ { "slug": "paper-iii-alignment-scaling-problem", "label": "Paper III" }, { "slug": "foundational", "label": "Foundational" } ], "decides": { "question": "Whether a prediction written down before the outcome was seen came out, with its forensic analysis.", "source": "the folder's own contents, read at source" }, "run_command": "", "run_note": "the folder holds no instructions file, no dependency list and no run command", "requirements": "Not stated anywhere in the folder.", "run_prose": "There is no way to run it as published, and what is missing is the instructions file, a dependency list and a run command. The folder holds one script of about forty kilobytes, a forensic analysis and a results file, and no instructions file at all. Start at the blind-prediction-test folder.", "runnable_today": { "verdict": "no", "label": "not runnable", "qualifier": null, "missing": [ "an instructions file", "a dependency list", "a run command" ] }, "evidence_kind": "implementation check", "evidence_kind_basis": "Its tier in the repository is supporting, the tier reserved for a step validated by mathematics or by computation and never for an empirical result. This grades what a run would be, because the folder cannot be run as published.", "pass_means": "Nothing, until those land.", "fail_means": "A reader can follow the script and see the outcome it wrote down, without any way to obtain either again.", "ceiling": "Its tier in the repository is supporting, the tier reserved for a step validated by mathematics or by computation and never for an empirical result.", "reproducibility_note": null, "settles_prose": "Nothing, until those land. A reader can follow the script and see the outcome it wrote down, without any way to obtain either again. Its tier in the repository is supporting, the tier reserved for a step validated by mathematics or by computation and never for an empirical result.", "settles_merged": [], "protocol_status": "Supporting", "caveat": null, "licence_key": "reserved", "instructions": { "present": false, "file": null, "blob_sha": null, "bytes": null, "secondary": [], "missing": [ "an instructions file", "a dependency list", "a run command" ] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "experiments/blind-prediction-test__Paper-III-and-Foundational", "iso": "2026-09-04T22:58:34Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "Paper III and Foundational, neither of which links it", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [ { "id": "FALS-009", "status": "open" }, { "id": "FALS-018", "status": "open" } ] } }, { "id": "T10", "group": "decidability", "order": 2, "title": "The decision-rule instruments kit", "index_name": "decision-rule instruments kit", "repository": "arc-principle-validation", "folder": "instruments/arc-instruments", "links": { "run": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/instruments/arc-instruments", "label": "the arc-instruments folder", "what": "the kit's own directory, holding the decision rules, the simulators and the README" }, "code": { "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/blob/main/instruments/README.md", "label": "instruments folder README", "what": "the folder README above the kit, which carries the install line, the recorded timing and the manifest reconciliation" } }, "papers": [ { "slug": "the-arc-theory", "label": "The ARC Theory" }, { "slug": "recursive-dynamics-founding-paper", "label": "Recursive Dynamics" } ], "decides": { "question": "Whether the deciding experiments can be told apart before they are run. The kit holds executable decision rules for the twenty-two propositions in the theory-level registration draft, design-sensitivity simulators for the deciding experiments, sealed-prediction and held-out partition tooling, and a runner that carries two of the charters as code.", "source": "the kit's README.md" }, "run_command": "python3 -m pip install -r requirements-test.txt; python3 -m pytest -q; python3 runner_battery.py ", "run_note": "taken from the kit's own directory", "requirements": "Python 3.9 with numpy and scipy. No API key, because the kit makes no network call of its own and a provider adapter is something a caller supplies.", "run_prose": "The instruments folder README gives, from the kit's own directory, `python3 -m pip install -r requirements-test.txt`, then `python3 -m pytest -q`; the battery evidence comes from `python3 runner_battery.py` with an output directory and a seed count. Needs Python 3.9 with numpy and scipy, and no key, because the kit makes no network call of its own and a provider adapter is something a caller supplies. Start at the arc-instruments folder.", "runnable_today": { "verdict": "yes", "label": "runnable today", "qualifier": "install, run and the recorded battery output are all given", "missing": [] }, "evidence_kind": "implementation check", "evidence_kind_basis": "A run exercises decision rules and simulators against the kit's own record and touches no real system.", "pass_means": "The instruments folder README records a run of 515 tests passing in five to nine minutes on pinned versions, so a run that matches shows the decision rules and the simulators behave as the kit says", "fail_means": "a run that does not shows the kit and its own record disagree.", "ceiling": "Neither is a result about any real system: that README states that the kit is reference code and a design instrument, that it is not a validated deciding instrument and not empirical evidence, and that nothing in the folder is a result about any real system.", "reproducibility_note": "The instruments folder README records that nine of eighty hashed source files differ from the battery run in comments, docstrings and two human-facing strings, and that four further files fall outside the manifest.", "settles_prose": "The instruments folder README records a run of 515 tests passing in five to nine minutes on pinned versions, so a run that matches shows the decision rules and the simulators behave as the kit says, and a run that does not shows the kit and its own record disagree. Neither is a result about any real system: that README states that the kit is reference code and a design instrument, that it is not a validated deciding instrument and not empirical evidence, and that nothing in the folder is a result about any real system. It also records that nine of eighty hashed source files differ from the battery run in comments, docstrings and two human-facing strings, and that four further files fall outside the manifest.", "settles_merged": [], "protocol_status": "Reference code and a design instrument; not a validated deciding instrument", "caveat": null, "licence_key": "reserved", "instructions": { "present": true, "file": "README.md", "blob_sha": "f8bacef341bd5b2cc39f4ff2ddf6433b00c5537f", "bytes": 71314, "secondary": [ { "file": "instruments/README.md", "blob_sha": "d36b46b6458b53bd3aa34672f8591e1d6c7e114a", "bytes": 9052, "role": "the folder README above the kit, carrying the install line, the recorded timing and the manifest reconciliation" } ], "missing": [] }, "verified": { "instructions": { "date": "2026-09-09", "method": "gh api contents, read only" }, "folder_date": { "date": "2026-09-09", "method": "gh api commits?path=, read only" } }, "folder_last_commit": [ { "path": "instruments/arc-instruments", "iso": "2026-09-07T10:23:18Z", "is_the_test_folder": true, "print": true } ], "papers_clause": "The ARC Theory and Recursive Dynamics, neither of which links it, and the second closes by inviting a reader to run the test that could kill it, which is the arc-instruments folder", "binding": { "failure_condition_ids": [], "prediction_ids": [], "same_paper_adjacency": [] } } ], "not_tests": [ { "id": "shared", "folder": "experiments/shared", "repository": "arc-principle-validation", "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/shared", "label": "The shared client wrapper", "sentence": "The shared client wrapper states that nothing in it produces a result of its own and that nothing in it should be cited as evidence.", "disclaimer": "Nothing in it produces a result of its own, and nothing in it should be cited as evidence.", "tier": "tooling", "folder_last_commit": "2026-09-04T22:58:34Z" }, { "id": "arc-scaling-lineage", "folder": "experiments/arc-scaling-lineage", "repository": "arc-principle-validation", "url": "https://github.com/MichaelDariusEastwood/arc-principle-validation/tree/main/experiments/arc-scaling-lineage", "label": "The scaling lineage folder", "sentence": "The scaling lineage folder holds four scripts written before the experiments were organised by paper, and states that nothing in it is current and that no result in any paper rests on those scripts.", "disclaimer": "Nothing in it is current, and no result in any paper rests on those scripts.", "tier": "superseded", "folder_last_commit": "2026-09-04T22:58:34Z" } ], "not_tests_heading": "Two folders listed and not counted", "not_tests_lede": "Both disclaim being tests in their own words, and they are listed so that nothing is hidden.", "tests_heading": "The tests", "standing_offer": "Completed attempts reach this register by email with the run artefacts; entry requires a checkable artefact, never a claim alone.", "rules": { "no_aggregate_favourable_count": "This file may not carry a count, rate or score of favourable outcomes. Counting tests is permitted, because it counts what exists. Counting passes is not, because no aggregate of favourable results constitutes validation of the programme.", "counts_derive_at_build": "No count of this register's own population may be written into any summary field, and no sentence in a summary field may state a fact about that population. The page and the homepage row derive every such count at build. A field that can hold a count will eventually hold a stale one.", "binding_is_a_fact_not_a_wish": "failure_condition_ids and prediction_ids are empty for every entry, because no published register names a test. They are not removed. The day a kill condition names a test folder, its identifier goes in the field and the page renders it as a binding without another edit. Until then the page shows same-paper adjacency and calls it adjacency. The sentence the page prints about this is derived from whether the two lists are empty, never typed.", "enforced_by": "scripts/build-replication-attempts.py validate_tests(), and scripts/check-rung-wording-parity.py for the favourable-count rule", "population_is_published_work": "An entry belongs in this register only when it is an experiment of a paper published on this site and its results are published. Removed under that rule on 9 September 2026: T11, the oversight assay at instruments/oversight-assay, which named no paper on this site and whose own README states that it is instrument development and not an ARC experiment, an alignment score, a real-model result or a registered instrument, with no collection from a real model included. The folder is unchanged at source and this register is what no longer lists it. Every count on the page and in the homepage row derives from this file at build, so a removal restates itself on both surfaces without a hand edit.", "run_invitation_names_the_run_version": "A run command here names the runner whose results are published, never a later version that has produced none. Where an instructions file names a later runner as canonical and no result in the repository comes from it, the entry says both: it points a reader at the runner behind the published results, and it records what the instructions file names and what the repository index says about that later runner. No sentence invites a run of a version that has never been run. Swept across every entry on 9 September 2026, not only the one an adversary named: each runner named in a run command or a run paragraph was matched against the entry-point inventory the coordination repository holds for the validation repository, which records for every runner whether it exists at the head and what results in the tree come from it. One entry matched the rule and was repointed, T03. Three runners named elsewhere carry no result in the tree and were left, each for a stated reason and none of them a later version standing in front of an earlier one that has results: per_scorer_check.py in T09, whose folder the repository itself grades tooling that produces no result of its own; test_coscaling.py in T13 and T16, whose outcome is a pass or a fail rather than a deposited file; and realmodel_coscaling_v2.py in T14, which the runbook itself gives as a self-test whose output that runbook says is not data, and whose family has results at the version before it and at the version after it. The other repository's three entries hold no versioned runner in this sense. The inventory is not in this repository, so the sweep is recorded here rather than enforced by a gate; a gate would need that inventory beside the register." } }