Files
confidence-engine/docs/experiment-57j78.md
T
robbond ce01e70010 tooling: add pre-anchored update-only mode to canonical harness
Add FIXTURE_MODE=updateOnly support that bypasses Start and sends the
committed fixture (tests/fixtures/pre-anchored-update-savings-realism.json)
directly as an Update request body through production HTTP route.

scripts/reproduce-multi-turn-investigation.mjs:
  - Added ESM imports for deterministic fixture loading (fs, fileURLToPath, path)
  - Added FIXTURE_PATH constant pointing to committed fixture
  - Added fixtureMode env-var selector and runUpdateOnlyMode() function
  - Validates ANSWER_2 before any live call (zero calls if missing)
  - Verifies single savings-realism anchor invariant on load
  - Preserves all hardened capture fields in pre-anchored mode
  - Normal-mode Start→Update chain preserved under guard clause

tests/reproduce-multi-turn-investigation.harness.test.js:
  - Added 7 new harness tests for pre-anchored scenarios (46 total, all pass)
  - Updated runPreAnchoredSimulation to persist rejectedProposalSnapshot on rejection
  - Added runPreAnchoredSimulationWithBlock() helper

docs/:
  - New docs/experiment-57j78.md with full apparatus description
  - Updated docs/current-handoff.md with 57J.78 section
2026-08-12 10:19:09 +01:00

93 lines
4.5 KiB
Markdown

# Experiment 57J.78 — Pre-Anchored Update-Only Mode (Committed)
**Branch:** `feature/semantic-action-contract-v0.23`
**Starting HEAD:** `9b7721c` (experiment: audit pre-anchored live apparatus)
**Commit message:** `tooling: add pre-anchored update-only mode to canonical harness`
## Objective
Eliminate the dependency on temporary uncommitted script modifications identified in audit 57J.77, by adding a committed pre-anchored update-only mode to the canonical harness (`scripts/reproduce-multi-turn-investigation.mjs`). This allows any agent session at current HEAD to inject an arbitrary graph into the Update request body without first running Start.
## Changes Made
### 1. scripts/reproduce-multi-turn-investigation.mjs (+177 lines)
Added:
- ESM imports (`fs`, `fileURLToPath`, `path`) for deterministic fixture loading
- `FIXTURE_PATH` constant pointing to `tests/fixtures/pre-anchored-update-savings-realism.json`
- `fixtureMode` env-var selector (default: undefined → normal mode)
- `runUpdateOnlyMode()` async function:
- Validates ANSWER_2 env-var exists before any live call
- Loads committed fixture from deterministic path
- Verifies single savings-realism anchor invariant
- Deep-copies fixture graph (no mutation of original)
- Skips Start entirely; sends exactly one Update via `postJson()` through production HTTP route
- Preserves all hardened capture fields (answerMeaning, updatedProposal, structuralActionRequired, selectedQuestion, persistent graph snapshot)
- Blocks on missing ANSWER_2 with zero live calls
- Reports rejection diagnostics identically to normal mode
### 2. tests/reproduce-multi-turn-investigation.harness.test.js (+183 lines)
Added 7 new harness tests:
- Blocked ANSWER_2 → zero calls, correct error message
- Accepted structuralActionRequired=true in capture
- Rejected snapshot preservation with structural linkage errors
- Exact ANSWER_2 body forwarding verification
- Pre-anchored rejected answerMeaning preservation
- Blocked mode verification (zero fixture load errors)
- Normal-mode isolation proof (accepted/rejected capture unchanged)
Updated `runPreAnchoredSimulation` mock to persist `rejectedProposalSnapshot` on rejection return values. Added `runPreAnchoredSimulationWithBlock()` helper.
## Evidence
| Test Suite | Pre-existing | New | Total | Result |
|------------|-------------|-----|-------|--------|
| Harness harness tests | 39 | 7 | 46 | ALL PASS (19ms) |
- No production code changed
- No Ollama calls made
- No live API calls made
- Normal-mode Start→Update chain preserved under guard
- Syntax validated via `node --check`
## Execution Commands
### Pre-anchored update-only mode:
```bash
FIXTURE_MODE=updateOnly \
ANSWER_2="I am unsure whether the projected office savings from the relocation are realistic." \
node scripts/reproduce-multi-turn-investigation.mjs
```
### Normal start→update mode (unchanged):
```bash
node scripts/reproduce-multi-turn-investigation.mjs
```
## Design Decisions
1. **Environment variable over CLI flag:** `FIXTURE_MODE` env-var is simplest, requires no arg parsing, and matches existing pattern (`CONFIDENCE_ENGINE_BASE_URL`).
2. **ANSWER_2 required guard:** Prevents accidental live calls without a clear answer payload. Zero calls made if missing.
3. **ESM imports for path resolution:** `fileURLToPath(import.meta.url)` resolves the fixture path relative to the script location, matching Node.js ESM best practices.
4. **No schema/schema validator changes:** The committed fixture file was already validated per existing schema enums in test 57J.74 (tests on lines 582-624 of the test file).
5. **Normal mode guard:** `fixtureMode !== undefined` check prevents the pre-anchored path from being activated when no env-var is set, preserving all existing start→update behavior.
## Verification
1. All 46 harness tests pass in under 20ms
2. No production code was modified
3. Syntax validated via `node --check`
4. Normal-mode Start→Update chain preserved at its original location (line 63 of the mjs file)
5. Pre-anchored mode explicitly documented with inline JSDoc comments
## Satisfies 57J.77 Recommendation
> "Add committed pre-anchored mode to the canonical harness by adding a single config flag that bypasses Start and reads the fixture into the Update request body, mirroring what runPreAnchoredSimulation() documents as its intended behaviour."
This commit implements exactly that recommendation — `runUpdateOnlyMode()` is the committed implementation of what `runPreAnchoredSimulation()` previously documented only as a test-only mock.