Add FIXTURE_MODE=updateOnly support that bypasses Start and sends the committed fixture (tests/fixtures/pre-anchored-update-savings-realism.json) directly as an Update request body through production HTTP route. scripts/reproduce-multi-turn-investigation.mjs: - Added ESM imports for deterministic fixture loading (fs, fileURLToPath, path) - Added FIXTURE_PATH constant pointing to committed fixture - Added fixtureMode env-var selector and runUpdateOnlyMode() function - Validates ANSWER_2 before any live call (zero calls if missing) - Verifies single savings-realism anchor invariant on load - Preserves all hardened capture fields in pre-anchored mode - Normal-mode Start→Update chain preserved under guard clause tests/reproduce-multi-turn-investigation.harness.test.js: - Added 7 new harness tests for pre-anchored scenarios (46 total, all pass) - Updated runPreAnchoredSimulation to persist rejectedProposalSnapshot on rejection - Added runPreAnchoredSimulationWithBlock() helper docs/: - New docs/experiment-57j78.md with full apparatus description - Updated docs/current-handoff.md with 57J.78 section
93 lines
4.5 KiB
Markdown
93 lines
4.5 KiB
Markdown
# Experiment 57J.78 — Pre-Anchored Update-Only Mode (Committed)
|
|
|
|
**Branch:** `feature/semantic-action-contract-v0.23`
|
|
**Starting HEAD:** `9b7721c` (experiment: audit pre-anchored live apparatus)
|
|
**Commit message:** `tooling: add pre-anchored update-only mode to canonical harness`
|
|
|
|
## Objective
|
|
|
|
Eliminate the dependency on temporary uncommitted script modifications identified in audit 57J.77, by adding a committed pre-anchored update-only mode to the canonical harness (`scripts/reproduce-multi-turn-investigation.mjs`). This allows any agent session at current HEAD to inject an arbitrary graph into the Update request body without first running Start.
|
|
|
|
## Changes Made
|
|
|
|
### 1. scripts/reproduce-multi-turn-investigation.mjs (+177 lines)
|
|
|
|
Added:
|
|
- ESM imports (`fs`, `fileURLToPath`, `path`) for deterministic fixture loading
|
|
- `FIXTURE_PATH` constant pointing to `tests/fixtures/pre-anchored-update-savings-realism.json`
|
|
- `fixtureMode` env-var selector (default: undefined → normal mode)
|
|
- `runUpdateOnlyMode()` async function:
|
|
- Validates ANSWER_2 env-var exists before any live call
|
|
- Loads committed fixture from deterministic path
|
|
- Verifies single savings-realism anchor invariant
|
|
- Deep-copies fixture graph (no mutation of original)
|
|
- Skips Start entirely; sends exactly one Update via `postJson()` through production HTTP route
|
|
- Preserves all hardened capture fields (answerMeaning, updatedProposal, structuralActionRequired, selectedQuestion, persistent graph snapshot)
|
|
- Blocks on missing ANSWER_2 with zero live calls
|
|
- Reports rejection diagnostics identically to normal mode
|
|
|
|
### 2. tests/reproduce-multi-turn-investigation.harness.test.js (+183 lines)
|
|
|
|
Added 7 new harness tests:
|
|
- Blocked ANSWER_2 → zero calls, correct error message
|
|
- Accepted structuralActionRequired=true in capture
|
|
- Rejected snapshot preservation with structural linkage errors
|
|
- Exact ANSWER_2 body forwarding verification
|
|
- Pre-anchored rejected answerMeaning preservation
|
|
- Blocked mode verification (zero fixture load errors)
|
|
- Normal-mode isolation proof (accepted/rejected capture unchanged)
|
|
|
|
Updated `runPreAnchoredSimulation` mock to persist `rejectedProposalSnapshot` on rejection return values. Added `runPreAnchoredSimulationWithBlock()` helper.
|
|
|
|
## Evidence
|
|
|
|
| Test Suite | Pre-existing | New | Total | Result |
|
|
|------------|-------------|-----|-------|--------|
|
|
| Harness harness tests | 39 | 7 | 46 | ALL PASS (19ms) |
|
|
|
|
- No production code changed
|
|
- No Ollama calls made
|
|
- No live API calls made
|
|
- Normal-mode Start→Update chain preserved under guard
|
|
- Syntax validated via `node --check`
|
|
|
|
## Execution Commands
|
|
|
|
### Pre-anchored update-only mode:
|
|
```bash
|
|
FIXTURE_MODE=updateOnly \
|
|
ANSWER_2="I am unsure whether the projected office savings from the relocation are realistic." \
|
|
node scripts/reproduce-multi-turn-investigation.mjs
|
|
```
|
|
|
|
### Normal start→update mode (unchanged):
|
|
```bash
|
|
node scripts/reproduce-multi-turn-investigation.mjs
|
|
```
|
|
|
|
## Design Decisions
|
|
|
|
1. **Environment variable over CLI flag:** `FIXTURE_MODE` env-var is simplest, requires no arg parsing, and matches existing pattern (`CONFIDENCE_ENGINE_BASE_URL`).
|
|
|
|
2. **ANSWER_2 required guard:** Prevents accidental live calls without a clear answer payload. Zero calls made if missing.
|
|
|
|
3. **ESM imports for path resolution:** `fileURLToPath(import.meta.url)` resolves the fixture path relative to the script location, matching Node.js ESM best practices.
|
|
|
|
4. **No schema/schema validator changes:** The committed fixture file was already validated per existing schema enums in test 57J.74 (tests on lines 582-624 of the test file).
|
|
|
|
5. **Normal mode guard:** `fixtureMode !== undefined` check prevents the pre-anchored path from being activated when no env-var is set, preserving all existing start→update behavior.
|
|
|
|
## Verification
|
|
|
|
1. All 46 harness tests pass in under 20ms
|
|
2. No production code was modified
|
|
3. Syntax validated via `node --check`
|
|
4. Normal-mode Start→Update chain preserved at its original location (line 63 of the mjs file)
|
|
5. Pre-anchored mode explicitly documented with inline JSDoc comments
|
|
|
|
## Satisfies 57J.77 Recommendation
|
|
|
|
> "Add committed pre-anchored mode to the canonical harness by adding a single config flag that bypasses Start and reads the fixture into the Update request body, mirroring what runPreAnchoredSimulation() documents as its intended behaviour."
|
|
|
|
This commit implements exactly that recommendation — `runUpdateOnlyMode()` is the committed implementation of what `runPreAnchoredSimulation()` previously documented only as a test-only mock.
|