3.8 KiB
Experiment 60B.4 — Decision Materiality Rule (prompt-only)
Branch: feature/decision-sufficiency-v0.26
Date: 2026-08-13
Status: Complete
Type: PROMPT-ONLY — Bounded instruction addition plus deterministic prompt tests. No live model calls.
Reasoning gap from 60B.3
60B.3 confirmed the engine has no independent decision-sufficiency rule. Rule 20 states:
"Return selectedQuestion as null only when no consequential unresolved unknown remains."
This defines when to return null but does not define what makes an unknown non-consequential. The term "consequential" is undefined at decision level. Live results (60B.1 vs 60B.2) show the model defaults to generic continuation when no explicit closing cue exists — even when option evidence is quantified on both sides.
The gap: uncertainty remains is always true during investigation. The prompt does not instruct the model to distinguish this from remaining uncertainty could materially change which option is preferred.
New prompt rule
Added section "Decision Sufficiency Rule" between Proposal Rules and Decision Option Structure Rules in lib/graph/prompt-builder.js:
An unresolved decision between options should not remain open merely because some uncertainty still exists.
Keep a decision context unresolved only when you can identify a specific unresolved factor that could materially change which option is preferred.
If the currently supported evidence is sufficient to distinguish the options and no such material unresolved factor remains, resolve the existing decision context and do not ask a generic continuation question.
Why this is domain-general
- No financial vocabulary (no £, $, payback, cost comparison)
- No relocation or industry-specific terms
- No numeric thresholds or calculation frameworks
- No keyword-based routing or taxonomic classification
- The three semantics apply to any decision between options with competing evidence:
- uncertainty alone is not sufficient reason to continue
- continuation requires a specific material factor that could change the preferred option
- when no such factor remains, resolve the existing decision context rather than asking a generic question
Focused test results
Command: npx vitest run tests/graph/prompt-builder.test.js
Result: PASS (86/86)
New materiality tests verify:
- Positive: uncertainty-alone-is-not-enough, specific-material-factor-required, could-change-preferred-criterion, resolve-when-no-material-factor, generic-continuation-discouraged, Rule-20-preserved, domain-generality
- Negative: no financial thresholds, no currency examples, no relocation examples, no automatic-resolution-when-better-looking, no new schema fields, no new node kinds
What remains unproven until live regression
- Stability — deterministic prompt tests confirm the instruction text is present and well-formed, but do not verify the model follows it consistently across repeated runs
- Cross-domain generalisation — single-prompt-test coverage does not prove the rule works outside the test cases' structural patterns
- Interaction with existing rules — no regression test confirms the materiality rule does not interfere with Rule 20, option structure rules, or the semantic-to-mutation contract
- Edge cases — decisions where multiple partially-material factors exist; decisions with equal evidence across options; ambiguous factor specificity
Production changes
| File | Change |
|---|---|
lib/graph/prompt-builder.js |
Added "Decision Sufficiency Rule" section (3 sentences, 3 semantics) |
| No schema changes | |
| No validator changes | |
| No question-selection code changes | |
| No provider integration changes |
Commit messages
Production/tests: feat(reasoning): add decision materiality rule
Documentation: docs: record decision materiality rule