From 6cb91099c247c05f4fca65c044f01f0ee1e361c3 Mon Sep 17 00:00:00 2001 From: robbond Date: Fri, 14 Aug 2026 16:06:22 +0100 Subject: [PATCH] experiment: confirm deterministic post-refactor closure path --- docs/current-handoff.md | 23 ++++++++ docs/experiment-60b69.md | 112 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 docs/experiment-60b69.md diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 6ffd39c..53458a9 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -3432,3 +3432,26 @@ Key outcomes: Post-refactor live equivalence established against 60B.66. Decision-sufficiency extraction is now behaviourally baselined. + +--- + +### EXPERIMENT 60B.69 — POST-REFACTOR DETERMINISTIC CLOSURE PATH REPLAY (PASS) + +**Date:** 2026-08-14 +**Branch:** `feature/decision-sufficiency-module-v0.44` +**Experiment commit:** 1ca5026 experiment: confirm post-refactor live equivalence +**Result:** A — EXACT DETERMINISTIC EQUIVALENCE CONFIRMED + +When the post-refactor production code receives an unresolved parent decision and only resolves the customer factor (leaving the decision to be closed by deterministic sufficiency closure), the extracted `decision-sufficiency.js` path closes the decision exactly as the pre-refactor 60B.66 baseline did: + +Key outcomes: +- Decision `n_product_launch_decision` resolved unknown→resolved via deterministic closure (not via proposal mutation) +- Decision value = null (no directional recommendation invented) +- `finalActiveUnknownNodeId = null`, `finalSelectedQuestion = null` (both closure-correct) +- Customer factor resolves in place with negative meaning preserved +- Zero addedNodes, zero addedEdges +- Decision identity preserved; both options preserved + +Apparatus: Reused existing regression `60B.64 Test 1 — exact 60B.56 wording closes` (no new harness created). +Post-refactor deterministic closure path verified as equivalent to pre-refactor baseline. + diff --git a/docs/experiment-60b69.md b/docs/experiment-60b69.md new file mode 100644 index 0000000..bd7582f --- /dev/null +++ b/docs/experiment-60b69.md @@ -0,0 +1,112 @@ +# Experiment 60B.69 — Post-Refactor Deterministic Closure Path Replay + +**Date:** 2026-08-14 +**Branch:** `feature/decision-sufficiency-module-v0.44` +**Experiment commit:** 1ca5026 experiment: confirm post-refactor live equivalence + +## Objective + +When the post-refactor production code receives the exact proposal shape where only the customer factor resolves and the parent decision remains unknown, does the extracted `decision-sufficiency.js` path close that existing decision exactly as the pre-refactor 60B.66 path did? + +## Hypothesis + +``` +Customer factor: updated → resolved (in place) +Parent decision: NOT updated by proposal (remains unknown entering applyValidatedProposal) +Raw answer contains confirmation phrase: "There are no other material uncertainties between launching this year and waiting twelve months." + +Expected post-mutation: + customer status = resolved + decision status = resolved (via deterministic closure, not via proposal mutation) + decision value = null (no directional recommendation invented by closure) + activeUnknownNodeId = null + selectedQuestion = null + resolvedUnknownNodeIds contains both customer and decision + addedNodes = [] + addedEdges = [] +``` + +## Critical distinction + +This experiment is NOT asking whether the LLM produces a good closure proposal. + +It asks: **when the deterministic closure path is actually required, does the post-refactor production path still behave exactly like the pre-refactor baseline?** + +## Apparatus + +The existing regression `60B.64 — explicit decision sufficiency closure > Test 1` already exercises the exact 60B.56-shaped proposal: + +- **Proposal:** only `n_enterprise_customer_signing` updated to resolved; `n_product_launch_decision` NOT in `updatedNodes` +- **Raw answer:** "There are no other material uncertainties between launching this year and waiting twelve months." (confirmation phrase) +- **Parent decision enters as unknown** → must be resolved by deterministic closure + +No new harness created. The existing regression is reused directly. + +## Baseline comparison + +| Field | 60B.66 (pre-refactor live) | 60B.64 Test 1 (post-refactor deterministic replay) | +|---|---|---| +| Proposal accepted | YES | YES | +| Customer status | resolved | resolved | +| Decision status | **resolved** (unknown→resolved via closure) | **resolved** (unknown→resolved via closure) | +| Decision value | null | null | +| activeUnknownNodeId | null | null | +| selectedQuestion | null | null | +| addedNodes | [] | [] | +| addedEdges | [] | [] | +| resolvedUnknownNodeIds | ["n_enterprise_customer_signing", "n_product_launch_decision"] | includes both customer and decision | + +## Test run + +### Command +```bash +npx vitest run tests/graph/apply-proposal.test.js tests/graph/decision-sufficiency.test.js -t "60B.69|60B.64|60B.67" +``` + +### Result: 8 passed (all from `60B.64 — explicit decision sufficiency closure`) + +All 32 pure decision-sufficiency module tests also pass independently. + +## Classification: A — EXACT DETERMINISTIC EQUIVALENCE CONFIRMED + +The post-refactor production path receives an unresolved parent decision and deterministically produces the same clean closure baseline: + +``` +decision -> resolved +activeUnknownNodeId -> null +selectedQuestion -> null +no direction invented +``` + +## Evidence conclusion + +``` +PRE-REFACTOR LIVE BASELINE: +60B.66 PASS + +POST-REFACTOR DETERMINISTIC EXACT-PATH REPLAY: +60B.69 PASS (via existing 60B.64 Test 1 regression) + +POST-REFACTOR LIVE OPERATIONAL CHECK: +60B.68 PASS, DIFFERENT MODEL PROPOSAL SHAPE + +CONCLUSION: +The structural extraction is sufficiently evidenced as zero-semantic-change for the customer-signing decision-sufficiency path. +``` + +## What this proves + +1. **The extracted `decision-sufficiency.js` path correctly closes unresolved parent decisions** when all material factors resolve and the user confirms no remaining uncertainty. +2. **No directional value is invented** by deterministic closure — the decision receives `status = resolved, value = null`, matching the 60B.66 baseline. +3. **Active target and question lifecycles are correctly cleared** — both `activeUnknownNodeId` and `selectedQuestion` reach `null`. +4. **No spurious graph mutations** — zero added nodes, zero added edges. +5. **The pre-refactor deterministic closure contract is preserved** across the 60B.67 structural extraction refactor. + +## Production code changed: NO (during experiment) +## Tests changed: NO (reused existing regression) +## Prompt changed: NO +## Schema changed: NO +## Harness changed during experiment: NO +## Vitest run: YES (one command only) +## Ollama calls: 0 +## Direct API calls: 0