experiment: choose material-factor question alignment

2026-08-13 07:57:26 +01:00
parent d1fe4ca087
commit 854c3aa002
2 changed files with 330 additions and 0 deletions
@@ -2740,3 +2740,7 @@ selectedQuestion: "What outcome would demonstrate enough value to justify contin
 - £5M/year not isolated from description text

 Experiment 60B.7 diagnosed why selectedQuestion targets a generic parent decision instead of the specific material unknown just created (n_client_retention). Architecture tracing revealed: the model's selectedQuestion is purely advisory — both nodeId and text are deterministically overridden by selectActiveUnknownCandidate + formulateQuestion in apply-proposal.js. The deterministic scorer scores all unresolved unknowns by text keywords + structural position with zero recency or materiality bonus. Validation accepts any structurally valid unresolved node without checking whether it matches the factor that justified continuation. Classification: E (multiple factors — A prompt alignment gap + B selection priority gap + D validation gap). Minimum missing distinction: B — a Continuation-Reason → Question-Target rule connecting the materiality reason to follow-up question targeting. Implementation readiness: A — ready for bounded implementation via prompt-only addition of one bridging rule. Status pending Rob's review.
+
+---
+
+Experiment 60B.8 compared three minimal mechanisms for preserving a material factor into final question selection (A: honour valid model-selected nodeId as preferred target, B: deterministic material-factor priority via scoring adjustment, C: explicit continuationTarget field). Analysis of actual code confirmed: (1) selectedQuestion.nodeId already means "candidate for next question" with full structural validation in place, (2) ignoring a valid model-selected node is semantically contradictory given the field's purpose, (3) existing validation (validateSelectedQuestion + isSelectableUnresolvedUnknown) covers all validity checks, and (4) existing fallback chain at lines 3418-3437 of apply-proposal.js handles the invalid-target case. Candidate A was chosen: honour valid model-selected nodeId as preferred target before deterministic scoring, with fallback to selectActiveImplementationCandidate when invalid. No new schema fields, no new validators, no new keyword logic. Smallest boundary: one Rule 172 clarification + one modification in buildSelectedQuestionResult. Implementation readiness: A — ready for bounded implementation. Status pending Rob's review.