experiment: test materiality rule against real unresolved factor

2026-08-13 07:41:09 +01:00
parent e8e6986d15
commit 721f1ccb6e
2 changed files with 232 additions and 0 deletions
@@ -2699,3 +2699,42 @@ The engine resolved the existing decision context independently. All three entit
 - Decision direction expressed: newValue = "Relocate to Manchester"

 **One live call at qwen-claude:latest on http://127.0.0.1:3000. No production code changed.**
+
+## Experiment 60B.6 — Materiality Rule Against Real Unresolved Factor (2026-08-13)
+
+Test whether the materiality rule keeps a decision open when one genuinely decision-changing uncertainty remains — the opposite boundary of 60B.5's over-closure case.
+
+**Fixture:** `tests/fixtures/pre-anchored-decision-options.json`
+**Answer (verbatim):** "We have now quantified the full financial impact of replacing the two senior engineers and the delivery delay at about £600,000 as a one-off relocation cost. Staying put costs us an extra £2 million every year. The remaining issue is our largest client: we do not yet know whether they would leave if we relocated, and losing them would cost us about £5 million per year."
+
+**One live call at qwen-claude:latest on http://127.0.0.1:3000.**
+
+### Result: Classification B — MATERIAL FACTOR RECOGNISED BUT STRUCTURE PARTIAL
+
+The engine correctly keeps the decision open because of the client-retention uncertainty, creates a first-class unknown node for it, and clearly attributes it to the Relocate option via a `may_cause` edge. However, three structural weaknesses prevent an A:
+
+1. **No observation node for known financial comparison.** Unlike 60B.5 (which created `n_fin_quantification`), 60B.6 did not extract the £600k and £2M/year figures into a new observation node. The quantified comparison survives only implicitly through the original option descriptions, not as structured evidence.
+2. **The £5M/year downside is text-only.** The description mentions "~£5M/year" but it is embedded in prose rather than isolated as a structured numeric consequence.
+3. **Selected question is generic despite known material factor.** The engine asks "What outcome would demonstrate enough value to justify continuing?" — the same generic phrasing from 60B.2 — when it has just created a specific client-retention unknown. It should have targeted that uncertainty directly (e.g., "Will the organisation's largest client leave if we relocate to Manchester?").
+
+### Structural details
+
+```
+updatedNodes: []
+resolvedUnknownNodeIds: []
+addedNodes: [n_client_retention — kind=unknown, status=unknown, label="Largest client retention uncertainty"]
+addedEdges: [n_client_retention → opt_relocate, relationship="may_cause"]
+selectedQuestion: "What outcome would demonstrate enough value to justify continuing?" (generic)
+```
+
+**What worked:**
+- Decision identity preserved (status remained unknown)
+- Both option identities preserved
+- Client-retention uncertainty created as a first-class unknown node
+- Client risk clearly attributed to Relocate via `may_cause` edge and `childIds` on the unknown node
+- No fabricated factors or over-closure
+
+**What was weak:**
+- Generic question despite having just identified a specific material factor
+- No observation node for the known financial comparison (unlike 60B.5)
+- £5M/year not isolated from description text