docs: record decision materiality rule

This commit is contained in:
2026-08-13 07:24:18 +01:00
parent 5ce5e7349b
commit b671681ddc
2 changed files with 72 additions and 0 deletions
+4
View File
@@ -2671,3 +2671,7 @@ Tested whether the Confidence Engine can independently recognise when quantified
**Diagnosis:** Classification C — NO SUFFICIENCY RULE + CONTINUATION BIAS **Diagnosis:** Classification C — NO SUFFICIENCY RULE + CONTINUATION BIAS
**Missing distinction:** MATERIALITY / DECISION-RELEVANCE RULE **Missing distinction:** MATERIALITY / DECISION-RELEVANCE RULE
---
Experiment 60B.4 implemented the minimal correction: added a domain-general decision-sufficiency rule to the prompt (not schema, not validator). The rule instructs: uncertainty alone is not sufficient reason to continue; continuation requires a specific material factor that could change the preferred option; when no such factor remains, resolve the existing decision context rather than asking a generic question. Prompt-level addition only — three sentences in a new "Decision Sufficiency Rule" section in prompt-builder.js. Deterministic tests confirm all five positive and six negative criteria pass (86/86). No live regression performed. Status pending Rob's review.
+68
View File
@@ -0,0 +1,68 @@
# Experiment 60B.4 — Decision Materiality Rule (prompt-only)
**Branch:** `feature/decision-sufficiency-v0.26`
**Date:** 2026-08-13
**Status:** Complete
**Type:** PROMPT-ONLY — Bounded instruction addition plus deterministic prompt tests. No live model calls.
## Reasoning gap from 60B.3
60B.3 confirmed the engine has no independent decision-sufficiency rule. Rule 20 states:
> "Return selectedQuestion as null only when no consequential unresolved unknown remains."
This defines *when* to return null but does not define what makes an unknown non-consequential. The term "consequential" is undefined at decision level. Live results (60B.1 vs 60B.2) show the model defaults to generic continuation when no explicit closing cue exists — even when option evidence is quantified on both sides.
The gap: **uncertainty remains** is always true during investigation. The prompt does not instruct the model to distinguish this from **remaining uncertainty could materially change which option is preferred**.
## New prompt rule
Added section "Decision Sufficiency Rule" between Proposal Rules and Decision Option Structure Rules in `lib/graph/prompt-builder.js`:
> An unresolved decision between options should not remain open merely because some uncertainty still exists.
>
> Keep a decision context unresolved only when you can identify a specific unresolved factor that could materially change which option is preferred.
>
> If the currently supported evidence is sufficient to distinguish the options and no such material unresolved factor remains, resolve the existing decision context and do not ask a generic continuation question.
## Why this is domain-general
- No financial vocabulary (no £, $, payback, cost comparison)
- No relocation or industry-specific terms
- No numeric thresholds or calculation frameworks
- No keyword-based routing or taxonomic classification
- The three semantics apply to any decision between options with competing evidence:
1. uncertainty alone is not sufficient reason to continue
2. continuation requires a specific material factor that could change the preferred option
3. when no such factor remains, resolve the existing decision context rather than asking a generic question
## Focused test results
**Command:** `npx vitest run tests/graph/prompt-builder.test.js`
**Result:** PASS (86/86)
New materiality tests verify:
- Positive: uncertainty-alone-is-not-enough, specific-material-factor-required, could-change-preferred-criterion, resolve-when-no-material-factor, generic-continuation-discouraged, Rule-20-preserved, domain-generality
- Negative: no financial thresholds, no currency examples, no relocation examples, no automatic-resolution-when-better-looking, no new schema fields, no new node kinds
## What remains unproven until live regression
1. **Stability** — deterministic prompt tests confirm the instruction text is present and well-formed, but do not verify the model follows it consistently across repeated runs
2. **Cross-domain generalisation** — single-prompt-test coverage does not prove the rule works outside the test cases' structural patterns
3. **Interaction with existing rules** — no regression test confirms the materiality rule does not interfere with Rule 20, option structure rules, or the semantic-to-mutation contract
4. **Edge cases** — decisions where multiple partially-material factors exist; decisions with equal evidence across options; ambiguous factor specificity
## Production changes
| File | Change |
|------|--------|
| `lib/graph/prompt-builder.js` | Added "Decision Sufficiency Rule" section (3 sentences, 3 semantics) |
| No schema changes | |
| No validator changes | |
| No question-selection code changes | |
| No provider integration changes | |
## Commit messages
Production/tests: `feat(reasoning): add decision materiality rule`
Documentation: `docs: record decision materiality rule`