From 7cfeee140be0db6faa4e0d71e89064490dcbf9a8 Mon Sep 17 00:00:00 2001 From: robbond Date: Fri, 14 Aug 2026 18:01:29 +0100 Subject: [PATCH] docs: record sufficiency confirmation question --- docs/current-handoff.md | 42 +++++++++++++++ docs/experiment-60b73.md | 112 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 154 insertions(+) create mode 100644 docs/experiment-60b73.md diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 13a34a5..5638c74 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -3507,3 +3507,45 @@ The no-confirmation guard works live post-refactor. Parent decision stays open w Inverses the question-target problem of 60B.71: when the guard correctly keeps the decision open, the next question should be a sufficiency-confirmation question — not a generic decision_threshold continuation. Currently it cannot distinguish "need to confirm no remaining factors" from "need to investigate an unresolved factor." **Status:** pending Rob's review. Full analysis in docs/experiment-60b72.md. + + +--- + +--- + +--- + +### Experiment 60B.73 — Missing Sufficiency Confirmation Question (IMPLEMENTATION) + +**Date:** 2026-08-14 +**Branch:** `feature/sufficiency-confirmation-question-v0.45` +**Parent:** 60B.72 (diagnosis ready for implementation) +**Type:** Bounded implementation + focused verification + +#### What Changed + +Added State B detection in `formulateQuestion()` — before strategy selection — that detects: +```text +unresolved decision target +AND zero remaining material factors +AND no explicit sufficiency confirmation in resolvedValues +``` + +Produces template `decision_threshold_sufficiency_confirmation` with question: +"Is there anything else material that could change which option is better?" + +#### Key Design Decisions + +- **Detection:** Transient, from existing state (node kind/status + hasRemainingMaterialFactors + resolvedValues check) +- **Template:** New key in existing family `decision_threshold` — no new question family +- **Plumbing:** Uses existing `context.resolvedValues[]` — no new parameters or schema changes +- **Scope:** Single early-return branch before selectInvestigationStrategy + +#### Focused Verification + +8 new tests (60B.73 Tests 1-8) all pass. All 50 question-formulator tests pass. + +Pre-existing apply-proposal failures (4) confirmed unrelated via git stash/re-run. + +#### Status +COMPLETE — production + tests committed, documentation recorded. diff --git a/docs/experiment-60b73.md b/docs/experiment-60b73.md new file mode 100644 index 0000000..073a1ab --- /dev/null +++ b/docs/experiment-60b73.md @@ -0,0 +1,112 @@ +# Experiment 60B.73 — Missing Sufficiency Confirmation Question (Implementation) + +**Date:** 2026-08-14 +**Branch:** `feature/sufficiency-confirmation-question-v0.45` +**Parent:** 60B.72 (diagnosis ready for implementation) +**Type:** Bounded implementation + focused verification + +## Objective + +Replace the generic decision_threshold question ("What outcome would demonstrate enough value to justify X?") with a focused sufficiency confirmation/discovery question when: + +```text +target is an unresolved decision +AND hasRemainingMaterialFactors(target.id, graph) === false +AND isUserConfirmationOfNoRemainingUncertainty(raw answer) === false +``` + +## Implementation Boundary + +Location: `formulateQuestion()` in `lib/graph/question-formulator.js` +Branch: Before `selectInvestigationStrategy()` call +Detection: Transient (no persisted state) + +### Detection Logic + +State B detected in `formulateQuestion` after `reasoningPatternSelection` and before strategy selection: + +```js +// Guarded to decision-pattern context only +if ( + reasoningPatternSelection.pattern === "decision" && + node.kind !== "unknown" && // not a factor — the target decision itself + node.status !== "known" && // still unresolved + node.status !== "resolved" && + node.status !== "contradicted" && + hasRemainingMaterialFactors(node.id, graph) === false +) { + const resolved = context.resolvedValues || []; + const hasConfirmation = resolved.some((v) => + isUserConfirmationOfNoRemainingUncertainty(v), + ); + if (!hasConfirmation) → sufficiency template +} +``` + +## New Template + +Key: `decision_threshold_sufficiency_confirmation` +Family: `decision_threshold` (existing family, no new family) +Question: "Is there anything else material that could change which option is better?" + +This question preserves both functions: +1. User can answer "No" to confirm sufficiency → closure proceeds +2. User can name another factor if one exists → that factor becomes next unknown + +## Test Coverage (60B.73 — 8 tests) + +| # | Scenario | Expected | +|---|----------|----------| +| 1 | Exact State B: unresolved decision, zero remaining factors, no confirmation | sufficiency template selected; generic threshold wording absent | +| 2 | Question allows missing-factor discovery | Contains "anything else material" and "could change which option is better" | +| 3 | Genuine remaining factor remains | Normal path preserved; NOT sufficiency template | +| 4 | Explicit sufficiency confirmation present | Normal path preserved; NOT sufficiency template | +| 5 | Non-decision unknown target | Unchanged normal behavior | +| 6 | Ordinary decision_threshold for unknown factors | `decision_threshold` family preserved | +| 7 | Resolved factor stays resolved (zero remaining) | State B triggers correctly | +| 8 | Decision identity preserved | Reason mentions material factors; node unchanged | + +## Behavioral Guardrails + +### Preserved (NOT changed): +- Target selection logic +- Decision closure rule (`shouldCloseDecision` in decision-sufficiency.js) +- Remaining-factor detection (`hasRemainingMaterialFactors`) +- Resolution semantics +- SelectedQuestion node identity +- Materiality determination +- Preferred option / recommendation +- Schema / provider / harness +- Existing factor-first question behavior + +### Not changed: +```text +new schema field → NO +new persisted graph state → NO +new question family → NO (uses existing decision_threshold) +prompt change → NO +broad answer plumbing → NO (uses existing resolvedValues context) +``` + +## Focused Verification + +Command: `npx vitest run tests/graph/question-formulator.test.js tests/graph/apply-proposal.test.js -t "60B.73|60B.64|decision_threshold"` + +Result: 16 passed (8 new + 8 regression/preserved) + +## Pre-existing Regressions (NOT introduced by this experiment) + +Four apply-proposal test failures confirmed pre-existing (verified via git stash/re-run): +1. "rejects selected question referencing resolved node" — validation not catching resolved ref +2-4. Question casing mismatch: expects lowercase, receives capitalized + +## WHAT IS NOW GUARANTEED + +When the active target is an unresolved decision with zero represented remaining material factors but no explicit sufficiency confirmation: +- Engine asks focused sufficiency confirmation/discovery question instead of generic threshold question +- Decision remains the active target (no target change) +- The question allows both "No" (confirm sufficiency) and factor discovery + +## WHAT REMAINS UNPROVEN + +The 60B.71 live no-confirmation case must still be rerun once after this implementation to prove the user-facing question changes from generic decision_threshold to focused sufficiency confirmation/discovery.