confidence-engine/docs/experiment-57j77.md

Experiment 57J.77 — Pre-Anchored Live Apparatus Audit

Branch: feature/semantic-action-contract-v0.23 Starting HEAD: 85fb2b4 (experiment: validate controlled structural no-op live) Experiment commit: pending (experiment: audit pre-anchored live apparatus)

Objective

Answer exactly:

What exact committed code path was used to perform the successful pre-anchored live updates reported in 57J.75 and the subsequent £2m test, and is that path reproducible from current HEAD without uncommitted script edits?

This is a read-only tooling/evidence audit. No Ollama calls. No live API calls. No code modifications.

Git Pre-Check

branch = feature/semantic-action-contract-v0.23
working tree = clean

Confirmed before audit.

Part 1 — Committed Apparatus Inventory

tests/fixtures/pre-anchored-update-savings-realism.json

Classification: FIXTURE ONLY

A valid deterministic situationGraph with:

• 2 nodes (1 state + 1 unknown/savings-realism)
• 1 edge (n_savings_realism → n_relocation_state, relationship=depends_on)
• activeUnknownNodeId = "n_savings_realism"
• All node/edge fields populated per existing schema enums

Provides the graph data. Does not execute anything.

PRE_ANCHORED_FIXTURE + runPreAnchoredSimulation() in test file

Classification: TEST-ONLY HELPER

Located at end of tests/reproduce-multi-turn-investigation.harness.test.js.

runPreAnchoredSimulation(cfg):

1. Copies initialGraph (or defaults to fixture) via JSON.parse(JSON.stringify())
2. Does NOT call Start
3. Creates a mock api.post() method that returns hardcoded JSON responses
4. Calls api.post("/api/cases/update", { situationGraph, previousQuestion, answer }) — but api.post is entirely in-memory with no HTTP client
5. Returns captured fields (answerMeaning, proposal, structuralActionRequired, selectedQuestion)

Critical finding: The internal api object returns mock/hardcoded JSON. It does not instantiate a fetch() or make any network calls. It is a synchronous simulator that mirrors what the harness prints, but cannot exercise the real production Update path.

scripts/reproduce-multi-turn-investigation.mjs at HEAD

Classification: PRODUCTION HARNESS PATH (standard mode only)

Structure:

• Line 38: const startResult = await postJson("/api/cases/start", { scenario: config.scenario }) — unconditional. Always called first.
• Lines 67–160: Bounded for loop over config.answers. Each iteration calls postJson("/api/cases/update", ...).
• No config flag, no fixtureMode, no pre-anchored path.
• config.maxUpdates = 2 (default), answers are positional.

Answer: Current HEAD does NOT support a committed pre-anchored/update-only mode. The script always makes one Start call and then up to maxUpdates Update calls. No configuration switch exists.

Git commits d77a1ff, 8184e05, 85fb2b4

Commit	Message	Files Added/Modified
d77a1ff	tooling: add pre-anchored update fixture	NEW tests/fixtures/pre-anchored-update-savings-realism.json (60 lines); MOD tests/reproduce-multi-turn-investigation.harness.test.js (+425 lines)
8184e05	docs: record pre-anchored update apparatus	NEW docs/experiment-57j74.md (121 lines); MOD docs/current-handoff.md (+12 lines)
85fb2b4	experiment: validate controlled structural no-op live	NEW docs/experiment-57j75.md (101 lines); MOD docs/current-handoff.md (+8 lines)

No commit ever modified scripts/reproduce-multi-turn-investigation.mjs to add pre-anchored mode.

Part 2 — Canonical Script Truth

Does current HEAD support a committed pre-anchored/update-only mode? NO

There is no config field, no CLI flag, and no branching logic in the committed script that enables bypassing Start or loading the pre-anchored fixture directly into an Update request body.

Part 3 — Test-Helper Truth

runPreAnchoredSimulation() classification: MOCKED TEST-ONLY PATH

The function's internal api.post() is a JavaScript closure that returns static objects. It does not:

• Import or use any fetch/Axios/http client
• Read from process.env.* for connection targets
• Make network I/O under any condition

It mirrors what the harness would print if it had a pre-anchored mode, but it is not the production Update path.

Part 4 — 57J.75 Execution Reconstruction

Was the successful live call made using only code committed before the run? UNPROVEN

The apparatus from 57J.74 (commits d77a1ff + 8184e05) consists of:

1. The JSON fixture file (data, not executable)
2. A test-only mock helper (simulator, not production invoker)
3. Two documentation files

Neither of these commits added pre-anchored mode to the canonical harness script (scripts/reproduce-multi-turn-investigation.mjs). Experiment 57J.74 explicitly states: "Harness restored: YES. Scenario, answers, and maxUpdates in scripts/reproduce-multi-turn-investigation.mjs are at canonical defaults."

Experiment 57J.75 records a live call that injected the fixture's graph into the Update request — but this required a harness path that was never committed. The most plausible reconstruction:

Execution classification: B — temporary uncommitted harness modification

The live test likely used a one-off script modification to scripts/reproduce-multi-turn-investigation.mjs (or another small wrapper) that:

1. Loaded the JSON fixture from tests/fixtures/pre-anchored-update-savings-realism.json
2. Bypassed the Start call
3. Sent the fixture graph directly as the Update request's situationGraph field

This modification was uncommitted and later reverted (consistent with 57J.74's statement that the harness was "restored" to canonical state before committing).

Behavioural observation validity: VALID — the model produced structuralActionRequired = false, zero mutation, HTTP 200 / update_applied. This was an actual production call, not simulated.

Apparatus reproducibility: NON-DURABLE — the code path that made the call is not in the committed repository at HEAD.

Part 5 — £2m Live Result Reconstruction

The £2M figure appears in experiment documentation as part of Answer 1 in 57J.53 ("roughly £2M annual savings on office overhead") and in the fixture scenario text ("reduce operating costs"). It is not independently documented as a separate live call.

Completed experiment: NO Committed apparatus used: UNPROVEN Within explicit call budget: UNPROVEN

The £2m answer appears to be part of the 57J.53 normal start→update chain (not a pre-anchored update). There is no separate committed record of a dedicated £2m pre-anchored live call.

Evidence status: INFORMAL OBSERVATION — embedded within multi-turn answers, not independently audited as a pre-anchored experiment.

Part 6 — Reproducibility Test (Code Inspection Only)

Could a fresh Claude session at current HEAD reproduce the 57J.75 pre-anchored live call using only committed files? PARTIAL

What is missing: A committed mechanism to bypass Start and inject an arbitrary graph into the Update request body. Specifically:

• The canonical harness script lacks any fixtureMode or updateOnly config option
• There is no documented command to execute pre-anchored mode
• The test helper (runPreAnchoredSimulation()) only simulates

Part 7 — Evidence Classification

57J.75 classification: B — VALID OBSERVATION, NON-DURABLE APPARATUS

Why: The behavioural result is confirmed (a real production call was made). However, the exact execution path that made it cannot be established from committed code alone because no committed harness mode supports injecting an arbitrary pre-anchored graph into the Update request without first running Start.

Part 8 — Next Tooling Boundary

Smallest next tooling boundary: A — add committed update-only mode to canonical harness

Why: Adding a single config flag (fixtureMode: "updateOnly") to scripts/reproduce-multi-turn-investigation.mjs that:

1. Skips the Start call when fixtureMode is set
2. Reads the JSON fixture into the Update request's situationGraph field
3. Preserves all existing behavior when fixtureMode is absent

This keeps changes minimal (one config field, one conditional branch) rather than introducing a separate harness tool.

Scope Compliance

• No Ollama calls made.
• No live API calls made.
• No production code modified.
• No harness/tooling modified.
• No prompt changed.
• No validator changed.
• No schema changed.
• Dev server not disturbed.