experiment: validate proposition-specific decision question live

2026-08-13 17:09:17 +01:00
parent 4767de30f7
commit 60036ac495
2 changed files with 147 additions and 0 deletions
@@ -2865,3 +2865,7 @@ Experiment 60B.20 was the live verification of bounded structural context admiss
 Experiment 60B.21 tested whether the full reasoning chain from 60B.20 generalises to a materially different decision domain (product launch timing). **Classification: B — MATERIALITY GENERALISES, TARGETING DOES NOT.** The model correctly identified the customer-signing factor as a first-class unknown (kind=unknown, status=unknown), kept the decision open for this specific material factor (£700k of £1.2M), and did not invent unrelated uncertainty. The node `n_customer_signing_status` was selected as the target. However, the final question text ("What would clarify the relevant customer, user, or value recipient in this situation?") is generic rather than specific to customer signing — unlike 60B.20 which produced "will our largest client leave if we relocate?" Option ownership shifted from may_cause (opt→unknown) to depends_on (decision→unknown with conditional attribution in description), preserving correct semantic ownership but with different structural encoding. One live call at qwen-claude:latest on http://192.168.1.111:11434. No production code changed. Status pending Rob's review.

 Experiment 60B.22 was a read-only diagnosis of why the correct material target (n_customer_signing_status) produced a generic final question ("What would clarify the relevant customer, user, or value recipient in this situation?") instead of a direct proposition question. **Root cause: C — FAMILY CLASSIFICATION TOO BROAD.** The regex at line 1162 of question-formulator.js (`/\b(audience|customer|user|buyer|stakeholder|recipient|who experiences)\b/`) matched on "customer" in the node's combined label+description, triggering early return to `decision_audience` family before any proposition-extraction logic could run. 60B.20 succeeded because its label contained "client" (not "customer"), allowing fallthrough to `decision_evidence_clarification` which properly detects interrogative labels. Minimum corrective boundary: E — NARROW CUSTOMER/VALUE FAMILY CLASSIFICATION. The trigger regex should require explicit audience-identity phrasing rather than any occurrence of "customer". Status pending Rob's review.
+
+---
+
+Experiment 60B.24 ran the exact product-launch live regression (same fixture as 60B.21) on the post-60B.23 branch to verify the proposition-specificity fix end-to-end. **Classification: A — LIVE QUESTION-SPECIFICITY FIX CONFIRMED.** The model produced `uncertain_enterprise_customer_signing` (kind=unknown, status=unknown) with depends_on edge from n_product_launch_decision and affects edge to opt_launch_this_year, preserving £700k significance in the node description. The final selected question ("whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k of the £1.2M expected annual revenue...") is proposition-specific — not generic audience wording. Core reasoning chain preserved, decision remains unresolved, no unrelated uncertainty invented. One live Ollama call at qwen-claude:latest on http://192.168.1.111:11434. No production code changed.