experiment: diagnose material-factor question targeting

2026-08-13 07:49:37 +01:00
parent 721f1ccb6e
commit d1fe4ca087
2 changed files with 265 additions and 0 deletions
@@ -2738,3 +2738,5 @@ selectedQuestion: "What outcome would demonstrate enough value to justify contin
 - Generic question despite having just identified a specific material factor
 - No observation node for the known financial comparison (unlike 60B.5)
 - £5M/year not isolated from description text
+
+Experiment 60B.7 diagnosed why selectedQuestion targets a generic parent decision instead of the specific material unknown just created (n_client_retention). Architecture tracing revealed: the model's selectedQuestion is purely advisory — both nodeId and text are deterministically overridden by selectActiveUnknownCandidate + formulateQuestion in apply-proposal.js. The deterministic scorer scores all unresolved unknowns by text keywords + structural position with zero recency or materiality bonus. Validation accepts any structurally valid unresolved node without checking whether it matches the factor that justified continuation. Classification: E (multiple factors — A prompt alignment gap + B selection priority gap + D validation gap). Minimum missing distinction: B — a Continuation-Reason → Question-Target rule connecting the materiality reason to follow-up question targeting. Implementation readiness: A — ready for bounded implementation via prompt-only addition of one bridging rule. Status pending Rob's review.