docs: record sufficiency confirmation question
This commit is contained in:
@@ -3507,3 +3507,45 @@ The no-confirmation guard works live post-refactor. Parent decision stays open w
|
||||
Inverses the question-target problem of 60B.71: when the guard correctly keeps the decision open, the next question should be a sufficiency-confirmation question — not a generic decision_threshold continuation. Currently it cannot distinguish "need to confirm no remaining factors" from "need to investigate an unresolved factor."
|
||||
|
||||
**Status:** pending Rob's review. Full analysis in docs/experiment-60b72.md.
|
||||
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
### Experiment 60B.73 — Missing Sufficiency Confirmation Question (IMPLEMENTATION)
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Branch:** `feature/sufficiency-confirmation-question-v0.45`
|
||||
**Parent:** 60B.72 (diagnosis ready for implementation)
|
||||
**Type:** Bounded implementation + focused verification
|
||||
|
||||
#### What Changed
|
||||
|
||||
Added State B detection in `formulateQuestion()` — before strategy selection — that detects:
|
||||
```text
|
||||
unresolved decision target
|
||||
AND zero remaining material factors
|
||||
AND no explicit sufficiency confirmation in resolvedValues
|
||||
```
|
||||
|
||||
Produces template `decision_threshold_sufficiency_confirmation` with question:
|
||||
"Is there anything else material that could change which option is better?"
|
||||
|
||||
#### Key Design Decisions
|
||||
|
||||
- **Detection:** Transient, from existing state (node kind/status + hasRemainingMaterialFactors + resolvedValues check)
|
||||
- **Template:** New key in existing family `decision_threshold` — no new question family
|
||||
- **Plumbing:** Uses existing `context.resolvedValues[]` — no new parameters or schema changes
|
||||
- **Scope:** Single early-return branch before selectInvestigationStrategy
|
||||
|
||||
#### Focused Verification
|
||||
|
||||
8 new tests (60B.73 Tests 1-8) all pass. All 50 question-formulator tests pass.
|
||||
|
||||
Pre-existing apply-proposal failures (4) confirmed unrelated via git stash/re-run.
|
||||
|
||||
#### Status
|
||||
COMPLETE — production + tests committed, documentation recorded.
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# Experiment 60B.73 — Missing Sufficiency Confirmation Question (Implementation)
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Branch:** `feature/sufficiency-confirmation-question-v0.45`
|
||||
**Parent:** 60B.72 (diagnosis ready for implementation)
|
||||
**Type:** Bounded implementation + focused verification
|
||||
|
||||
## Objective
|
||||
|
||||
Replace the generic decision_threshold question ("What outcome would demonstrate enough value to justify X?") with a focused sufficiency confirmation/discovery question when:
|
||||
|
||||
```text
|
||||
target is an unresolved decision
|
||||
AND hasRemainingMaterialFactors(target.id, graph) === false
|
||||
AND isUserConfirmationOfNoRemainingUncertainty(raw answer) === false
|
||||
```
|
||||
|
||||
## Implementation Boundary
|
||||
|
||||
Location: `formulateQuestion()` in `lib/graph/question-formulator.js`
|
||||
Branch: Before `selectInvestigationStrategy()` call
|
||||
Detection: Transient (no persisted state)
|
||||
|
||||
### Detection Logic
|
||||
|
||||
State B detected in `formulateQuestion` after `reasoningPatternSelection` and before strategy selection:
|
||||
|
||||
```js
|
||||
// Guarded to decision-pattern context only
|
||||
if (
|
||||
reasoningPatternSelection.pattern === "decision" &&
|
||||
node.kind !== "unknown" && // not a factor — the target decision itself
|
||||
node.status !== "known" && // still unresolved
|
||||
node.status !== "resolved" &&
|
||||
node.status !== "contradicted" &&
|
||||
hasRemainingMaterialFactors(node.id, graph) === false
|
||||
) {
|
||||
const resolved = context.resolvedValues || [];
|
||||
const hasConfirmation = resolved.some((v) =>
|
||||
isUserConfirmationOfNoRemainingUncertainty(v),
|
||||
);
|
||||
if (!hasConfirmation) → sufficiency template
|
||||
}
|
||||
```
|
||||
|
||||
## New Template
|
||||
|
||||
Key: `decision_threshold_sufficiency_confirmation`
|
||||
Family: `decision_threshold` (existing family, no new family)
|
||||
Question: "Is there anything else material that could change which option is better?"
|
||||
|
||||
This question preserves both functions:
|
||||
1. User can answer "No" to confirm sufficiency → closure proceeds
|
||||
2. User can name another factor if one exists → that factor becomes next unknown
|
||||
|
||||
## Test Coverage (60B.73 — 8 tests)
|
||||
|
||||
| # | Scenario | Expected |
|
||||
|---|----------|----------|
|
||||
| 1 | Exact State B: unresolved decision, zero remaining factors, no confirmation | sufficiency template selected; generic threshold wording absent |
|
||||
| 2 | Question allows missing-factor discovery | Contains "anything else material" and "could change which option is better" |
|
||||
| 3 | Genuine remaining factor remains | Normal path preserved; NOT sufficiency template |
|
||||
| 4 | Explicit sufficiency confirmation present | Normal path preserved; NOT sufficiency template |
|
||||
| 5 | Non-decision unknown target | Unchanged normal behavior |
|
||||
| 6 | Ordinary decision_threshold for unknown factors | `decision_threshold` family preserved |
|
||||
| 7 | Resolved factor stays resolved (zero remaining) | State B triggers correctly |
|
||||
| 8 | Decision identity preserved | Reason mentions material factors; node unchanged |
|
||||
|
||||
## Behavioral Guardrails
|
||||
|
||||
### Preserved (NOT changed):
|
||||
- Target selection logic
|
||||
- Decision closure rule (`shouldCloseDecision` in decision-sufficiency.js)
|
||||
- Remaining-factor detection (`hasRemainingMaterialFactors`)
|
||||
- Resolution semantics
|
||||
- SelectedQuestion node identity
|
||||
- Materiality determination
|
||||
- Preferred option / recommendation
|
||||
- Schema / provider / harness
|
||||
- Existing factor-first question behavior
|
||||
|
||||
### Not changed:
|
||||
```text
|
||||
new schema field → NO
|
||||
new persisted graph state → NO
|
||||
new question family → NO (uses existing decision_threshold)
|
||||
prompt change → NO
|
||||
broad answer plumbing → NO (uses existing resolvedValues context)
|
||||
```
|
||||
|
||||
## Focused Verification
|
||||
|
||||
Command: `npx vitest run tests/graph/question-formulator.test.js tests/graph/apply-proposal.test.js -t "60B.73|60B.64|decision_threshold"`
|
||||
|
||||
Result: 16 passed (8 new + 8 regression/preserved)
|
||||
|
||||
## Pre-existing Regressions (NOT introduced by this experiment)
|
||||
|
||||
Four apply-proposal test failures confirmed pre-existing (verified via git stash/re-run):
|
||||
1. "rejects selected question referencing resolved node" — validation not catching resolved ref
|
||||
2-4. Question casing mismatch: expects lowercase, receives capitalized
|
||||
|
||||
## WHAT IS NOW GUARANTEED
|
||||
|
||||
When the active target is an unresolved decision with zero represented remaining material factors but no explicit sufficiency confirmation:
|
||||
- Engine asks focused sufficiency confirmation/discovery question instead of generic threshold question
|
||||
- Decision remains the active target (no target change)
|
||||
- The question allows both "No" (confirm sufficiency) and factor discovery
|
||||
|
||||
## WHAT REMAINS UNPROVEN
|
||||
|
||||
The 60B.71 live no-confirmation case must still be rerun once after this implementation to prove the user-facing question changes from generic decision_threshold to focused sufficiency confirmation/discovery.
|
||||
Reference in New Issue
Block a user