experiment: audit pre-anchored live apparatus

2026-08-12 09:56:11 +01:00
parent 85fb2b4256
commit 9b7721c610
2 changed files with 188 additions and 0 deletions
@@ -1449,3 +1449,26 @@ New branch: `feature/semantic-action-contract-v0.23`
 **Answer:** The one controlled run shows that with an equivalent savings-realism uncertainty already present, the model directly emitted `structuralActionRequired = false`, produced zero meaningful mutation (no added/updated nodes or edges), and the production validator accepted the proposal. All four required conditions met: structuralActionRequired=false observed directly, zero meaningful mutation, HTTP 200 / update_applied, exactly one equivalent savings-realism uncertainty remained after Update.

 **What remains unproven:** Whether `false + zero mutation` would also be accepted for non-anchored graphs; cold-start comparison via normal Start→Update; stability across repeated runs (only one run); generalization beyond savings-realism anchors.
+
+---
+
+### Experiment 57J.77 — Pre-Anchored Live Apparatus Audit
+
+**Objective:** Determine whether the live pre-anchored Update call reported in 57J.75 was executed through committed, reproducible apparatus — or via temporary uncommitted script modifications — and classify evidence durability. **Classification: B — VALID OBSERVATION, NON-DURABLE APPARATUS.**
+
+**Key finding:** The canonical harness (`scripts/reproduce-multi-turn-investigation.mjs`) at current HEAD supports only the standard `start → update(n)` path. It has no pre-anchored mode, no fixtureMode config, and makes one unconditional start call on every invocation (line 38). The `runPreAnchoredSimulation()` helper in the test file is a synchronous mock simulator whose internal API object returns hardcoded responses — it does not make real HTTP calls to production.
+
+The 57J.74 apparatus commit ("docs: record pre-anchored update apparatus") explicitly states: "Harness restored: YES. Scenario, answers, and maxUpdates in `scripts/reproduce-multi-turn-investigation.mjs` are at canonical defaults." No fixtureMode was committed to the harness script.
+
+The 57J.75 experiment document ("experiment: validate controlled structural no-op live") records a live call that "sends the fixture's graph directly as the Update request's situationGraph" — but this capability does not exist in the committed mjs script. The live test depended on temporary uncommitted harness modifications (or direct manual production API invocation) at the time of execution. These were reverted before the apparatus was committed (which only added the fixture file and test-only helper).
+
+**Apparatus classification:**
+- `tests/fixtures/pre-anchored-update-savings-realism.json`: FIXTURE ONLY — valid deterministic graph, committed
+- `PRE_ANCHORED_FIXTURE` constant + `runPreAnchoredSimulation()`: TEST-ONLY HELPER — synchronous mock simulator in test file, never makes real HTTP calls
+- `scripts/reproduce-multi-turn-investigation.mjs`: PRODUCTION HARNESS PATH — but only standard start→update chain, no pre-anchored mode
+
+**57J.75 classification: B.** The behavioural observation (model produces false + zero mutation → accepted on a pre-anchored graph) is valid and was confirmed by an actual production call. However, the execution route cannot be established from committed code alone — it depended on uncommitted harness modifications made for that specific run and later reverted.
+
+**What remains unproven:** How the 57J.75 live call was actually sent (uncommitted script edit vs. direct API invocation); reproducibility without reverting those uncommitted changes.
+
+**Smallest next tooling boundary: A.** Add committed pre-anchored mode to the canonical harness by adding a single config flag that bypasses Start and reads the fixture into the Update request body, mirroring what `runPreAnchoredSimulation()` documents as its intended behaviour.