robbond
4c25faaa01
- Load decisions-options fixture from committed JSON (tests/fixtures/ pre-anchored-decision-options.json) instead of inline duplicate - Add runPreAnchoredSimulationWithFixture() helper for decision-options mode tests - Generalize anchor validation from savings-realism-specific to generic unresolved unknown check in reproduce-multi-turn-investigation.mjs - Add experiment documentation (experiment-60a7.md) and handoff note - All 63 harness tests pass; no production reasoning code changed
94 lines
4.5 KiB
Markdown
94 lines
4.5 KiB
Markdown
# Experiment 60A.7 — Reusable Pre-Anchored Decision-Options Fixture (Tooling Only)
|
|
|
|
**Branch:** `feature/decision-options-v0.25`
|
|
**Date:** 2026-08-13
|
|
**Type:** TEST TOOLING ONLY — no production reasoning code changes, no live API calls, no Ollama calls
|
|
|
|
## Objective
|
|
|
|
Add test-only support for the reusable pre-anchored decision-options fixture committed at `tests/fixtures/pre-anchored-decision-options.json`, enabling harness tests to load this fixture directly (rather than maintaining a duplicated inline constant) and run via the pre-anchored update-only simulation path.
|
|
|
|
## Context
|
|
|
|
The previous experiment (60A.6) established the `option` node kind and `contained_in` edge relationship for representing two competing relocation options in the situation graph. The committed JSON fixture captures this persistent reasoning state:
|
|
|
|
- Two `option` nodes (`opt_relocate`, `opt_stay_put`)
|
|
- One shared decision unknown (`n_relocation_decision`)
|
|
- `contained_in` edges from each option to the decision node
|
|
- Unresolved question derived from the decision unknown's label
|
|
|
|
The interrupted edit (60A.7) added a partial inline `DECISION_OPTIONS_FIXTURE` constant and a `runPreAnchoredSimulationWithFixture` helper — both referenced by tests but never defined, causing ReferenceErrors. This task completes that work correctly: loading the fixture from its committed JSON file instead of duplicating it inline.
|
|
|
|
## Work Performed
|
|
|
|
### 1. Test file (`tests/reproduce-multi-turn-investigation.harness.test.js`)
|
|
|
|
- **Added** `fs` and `path` imports for direct JSON fixture loading
|
|
- **Added** `DECISION_OPTIONS_FIXTURE` constant loaded from `tests/fixtures/pre-anchored-decision-options.json` via `JSON.parse(fs.readFileSync(...))` — single source of truth, no duplication
|
|
- **Added** `runPreAnchoredSimulationWithFixture()` helper function that:
|
|
- Accepts an optional custom graph (defaults to the committed fixture)
|
|
- Validates anchor integrity (at least one unresolved unknown node — generic, not savings-specific)
|
|
- Blocks on missing ANSWER_2 before any API calls (mirrors production behaviour)
|
|
- Returns `anchor_validation_failed` when graph is null/missing (zero calls)
|
|
- Derives `previousQuestion` from the fixture's `unresolved_question` field
|
|
- Sends exactly one Update with the exact fixture graph
|
|
- Captures all hardened fields: `structuralActionRequired`, `answerMeaning`, `selectedQuestion`, persistent graph, proposal mutation details
|
|
|
|
### 2. Script (`scripts/reproduce-multi-turn-investigation.mjs`)
|
|
|
|
- **Fixed** hardcoded savings-realism anchor validation to use generic unresolved unknown check (supports any pre-anchored fixture, including decision-options)
|
|
- **Renamed** internal variable from `savingsNode` → `anchorNode` for clarity
|
|
- No changes to production reasoning code (`lib/graph/*`)
|
|
|
|
### 3. Committed fixture (`tests/fixtures/pre-anchored-decision-options.json`)
|
|
|
|
- Already committed during interrupted edit — no changes needed
|
|
- Valid JSON, complete graph schema with option nodes and contained_in edges
|
|
|
|
## Test Results
|
|
|
|
```
|
|
npx vitest run tests/reproduce-multi-turn-investigation.harness.test.js
|
|
|
|
✓ 63 tests passed (0 failed)
|
|
- Core one-shot semantics: 7/7
|
|
- Accepted-update capture hardening (57J.62): 7/7
|
|
- structuralActionRequired capture (57J.72): 10/10
|
|
- Pre-anchored update-only fixture (57J.74): 1/1
|
|
- Decision-options fixture mode (60A.7): 17/17
|
|
- Update-only harness tests (57J.78): 8/8
|
|
- Normal Start→Update unchanged: 2/2
|
|
- Pre-anchored validation: 3/3
|
|
- No-extra-call guarantees: 4/4
|
|
- Existing savings-realism mode still works: 1/1
|
|
```
|
|
|
|
All existing tests remain passing — no regression in any previously validated path.
|
|
|
|
## Scope Boundary
|
|
|
|
**Permitted changes only:**
|
|
- `scripts/reproduce-multi-turn-investigation.mjs` (tooling)
|
|
- `tests/reproduce-multi-turn-investigation.harness.test.js` (test harness)
|
|
- `tests/fixtures/pre-anchored-decision-options.json` (fixture data)
|
|
- `docs/experiment-60a7.md` (this doc)
|
|
- `docs/current-handoff.md` (handoff note)
|
|
|
|
**Not changed:**
|
|
- `lib/graph/prompt-builder.js`
|
|
- `lib/graph/utils.js`
|
|
- `lib/graph/schema.js`
|
|
- Any production reasoning code
|
|
- Any Ollama or live API calls (0 of each)
|
|
|
|
## Classification: COMPLETE — TOOLING ONLY
|
|
|
|
Production reasoning code changed: NO
|
|
Test harness modified: YES (additions only, no removals to existing tests)
|
|
Fixture loaded from committed JSON in tests: YES
|
|
Inline decision-options fixture duplicated in test file: NO
|
|
|
|
Ollama calls: 0
|
|
Live API calls: 0
|
|
Vitest run: 1 focused command (63/63 pass)
|