# Experiment 60B.8 — Which minimal mechanism preserves the material factor into final question selection? **Branch:** `feature/decision-sufficiency-v0.26` **Date:** 2026-08-13 **Type:** READ-ONLY DESIGN COMPARISON — Three candidate mechanisms evaluated against actual codebase structure and semantics. --- ## Context Experiment 60B.7 proved that: ``` reasoning layer: correctly identifies a specific material unresolved factor (n_client_retention) deterministic selection: ignores the model-selected nodeId and independently re-selects from all unresolved unknowns selectedQuestion text: generic template matched to the parent decision node, not the material factor ``` The gap is purely at the deterministic-selection layer: a valid model-selected nodeId exists but carries zero weight. Three mechanisms are assessed against this exact gap. --- ## Existing selectedQuestion semantics (before any change) ### Does model-selected selectedQuestion.nodeId already mean "this is the unresolved node we should ask about next"? **PARTIAL.** Rule 16 requires the model to select "one of those" — not specifically the material factor. Rule 17 constrains it structurally. Rule 172 explicitly states: *"the engine retains deterministic final-priority selection and may choose a different question if multiple candidates exist."* The field communicates *candidate intent*, not binding assignment. However, when the model intentionally picks one of its own newly-created nodes (as in 60B.6), the intent clearly points to that node as the intended target. ### Is ignoring it in final deterministic selection semantically contradictory? **PARTIAL.** Contractually: no. Rule 172 grants override authority. Practically: yes, when the model's nodeId is one of its own newly-created unknowns. The contract gives the engine authority to override, but there is zero semantic justification for overriding a valid model-selected newly-added node — the engine has no information the model lacks about which factor matters for this decision. ### Can current validation already establish that the proposed nodeId: - exists or is added in this proposal - is kind=unknown - is unresolved **YES.** All three checks are in `validateSelectedQuestion` (line 215): - `buildNodeById(graph, proposal.addedNodes)` includes newly-added nodes - `node.kind !== "unknown"` check at line 232 - `resolvesNode || effectiveStatus === "resolved"` check at lines 238-250 ### Can deterministic formulation safely generate text from that node after mutation? **YES.** After `applyGraphUpdate` (line 3358), `findNodeById(updatedSituationGraph, nodeId)` in `buildSelectedQuestionResult` (line 2110) retrieves the node. `formulateQuestion` generates deterministic text from templates. The pipeline is intact — no structural barrier exists. --- ## Candidate A — Honour Valid Model-Selected Node as Preferred Target ### Concept If `proposal.selectedQuestion.nodeId` passes existing validation: ``` selectedNode is structurally valid (exists/is-added, kind=unknown, unresolved) ``` Use it as the **preferred target** in `buildSelectedQuestionResult`. If the preferred target disappears between preference and finalisation (e.g., another proposal mutation resolves it), fall back to `selectActiveUnknownCandidate(...)`. ### Mechanism sketch (conceptual only — no code) In `buildSelectedQuestionResult`: ``` if proposedTarget is valid AND still unresolved in updatedSituationGraph: use proposedTarget as selectedNode formulate question for it else: run selectActiveUnknownCandidate(updatedSituationGraph, resolvedNodeIds) ``` The model-selected nodeId already carries the correct semantic meaning when it points to a newly-added unknown. No new field, no new schema, no new keyword logic. ### Materiality fidelity: HIGH When the model identifies `n_client_retention` as its selectedQuestion and it passes validation, that node becomes the question target. The material factor is preserved through the full pipeline. ### Deterministic safety: HIGH Validation is already in place — `validateSelectedQuestion` covers existence, kind=unknown, unresolved status. Post-mutation, `isSelectableUnresolvedUnknown` provides a second check with identical semantics. Fallback to existing `selectActiveUnknownCandidate` is guaranteed when the preferred target becomes invalid. No new failure path; the existing fallback already exists at line 3431-3436 (`if !remainingUnknownExists → run selectActiveUnknownCandidate`). ### Semantic honesty: HIGH `selectedQuestion.nodeId` already means "candidate for next question". Using it as a preferred target is a semantic natural extension — not an overload. The existing prompt Rule 172 would need a minor clarification (the engine has priority authority, but prefers the model's choice when valid), but the field itself is used for its intended purpose. ### New schema fields: None. Uses `selectedQuestion.nodeId` only. ### New validator rules: None. Existing `validateSelectedQuestion` + `isSelectableUnresolvedUnknown` are sufficient. ### New prompt concepts: Minor clarification of Rule 172 to indicate that when the model selects a valid unresolved unknown (especially from addedNodes), the engine prefers that selection as the preferred target. Not a new rule — an intensification of existing contract language. ### New scoring logic: None. The preferred target does not enter `scoreUnknownCandidate`. It short-circuits scoring entirely when valid, then falls through to scoring only when invalid/lost. ### Principal risk: Model selects a valid but low-value unknown (e.g., a tangentially-related newly-created node rather than the truly material one). **Mitigation:** This risk already exists today — deterministic selection also sometimes picks a lower-value target based on keyword matching. The key difference is that today's fallback picks without any information about what matters; Candidate A's fallback only activates when the preferred target becomes structurally invalid, which is rare in practice (same-turn mutations rarely resolve another proposal's selected node). ### 60B.6 final nodeId: `n_client_retention` The model would select it in its proposal → validation passes (exists in addedNodes, kind=unknown, unresolved) → preferred target honoured → question formulated for `n_client_retention`. --- ## Candidate B — Deterministic Material-Factor Priority ### Concept Adjust deterministic selection (`selectActiveUnknownCandidate` or scoring within it) so that newly-created unresolved factors associated with the current decision get priority over their parent decision. ### Materiality fidelity: MEDIUM Could partially recover the material factor through structural heuristics (e.g., preferring nodes added in the same proposal, nodes whose `affects` edges target options), but only if distinguishing "material child" from "incidental unknown" requires domain-specific logic — which contradicts criterion #4. ### Deterministic safety: MEDIUM Would require either: - New heuristic rules in scoring (introduces brittleness) - Recency-based ranking ("newest unresolved wins") which fails when multiple unrelated new unknowns are created - Structural edge analysis to determine "material relevance" which requires keyword/schema logic ### Semantic honesty: MEDIUM No new schema. But the mechanism would need to inject domain knowledge (e.g., "prefer nodes connected via `may_cause` edges to options") that doesn't exist in the current scoring model. ### New schema fields: None. ### New validator rules: Would require new scoring heuristics — a form of implicit validation logic. ### New prompt concepts: Would need rule change explaining why deterministic selection now behaves differently for newly-added nodes (recency or structural priority). ### New scoring logic: New rules needed to distinguish "material child" from "incidental unknown". This is the core problem: there is no existing signal that identifies materiality. Any mechanism would need domain-specific keyword analysis of edge types, description patterns, or dependency chains. ### Principal risk: Recency bias mistakes incidental nodes for material factors when the model creates multiple unrelated new unknowns (max 3 per Rule 8). The scoring function would need to guess at "importance" without a reliable signal — precisely what experiment 60B.7 identified as the root cause of the problem. ### 60B.6 final nodeId: Unpredictable by structural metrics alone Without knowing whether `n_client_retention` or `n_relocation_decision` scores higher on keyword density + downstream count, we cannot guarantee it would be selected. The parent decision (`n_relocation_decision`) may accumulate higher score from accumulated label/description text patterns across the decision history. --- ## Candidate C — Explicit continuationTarget Field ### Concept Introduce a new proposal field `continuationTargetNodeId` conceptually distinct from `selectedQuestion.nodeId`: ``` continuationTargetNodeId: "the specific unresolved factor whose materiality justifies keeping the decision open" ``` Deterministic code validates and uses it as the preferred target. ### Materiality fidelity: HIGH Explicitly carries the material factor signal. No ambiguity about intent. ### Deterministic safety: HIGH New validation rule: `continuationTargetNodeId` must pass same checks as `selectedQuestion.nodeId`. But this adds surface area for new failure modes (null continuationTarget, stale target, mismatched semantics with selectedQuestion). ### Semantic honesty: LOW `selectedQuestion.nodeId` already means "candidate for next question". A new field duplicates its semantic space and creates confusion about which field controls the outcome. The model would need to produce two different fields that arguably carry the same information. ### New schema fields: `continuationTargetNodeId` — requires proposal shape change, schema migration in `situationGraphSchema`, prompt rule additions. ### New validator rules: New validation for the new field plus reconciliation logic with `selectedQuestion.nodeId`. ### New prompt concepts: New rule explaining when to populate `continuationTargetNodeId` vs `selectedQuestion.nodeId`, and what each means. ### New scoring logic: None directly, but validation and orchestrator would need awareness of the new field's semantics. ### Principal risk: New field duplicates `selectedQuestion` semantics, creating confusion about which field controls outcome. The model may populate only one (breaking the invariant), or populate both with different values (ambiguity). This adds contract surface without solving a semantic gap — `selectedQuestion.nodeId` already carries the correct information; it just gets ignored. ### 60B.6 final nodeId: `n_client_retention` If populated correctly. But the risk is model produces `continuationTargetNodeId` inconsistently across providers/runs, creating brittle dependency on reliable model behaviour for a field that has no fallback contract. --- ## Fallback Assessment ### Can valid preferred target → deterministic formulator → existing selector fallback work? **YES.** The existing pipeline already handles this pattern: 1. `validateSelectedQuestion` validates the proposed nodeId (exists, kind=unknown, unresolved) — lines 215-280 2. `applyGraphUpdate` applies mutations — line 3358 3. `isSelectableUnresolvedUnknown` checks still-valid status post-mutation — line 1658 4. `buildSelectedQuestionResult` retrieves node from graph — line 2110 5. If preferred target is invalid/lost, existing fallback at lines 3431-3436 runs `selectActiveUnknownCandidate` The entire chain exists. No new validation or graph semantics required. ### Requires graph semantic change? **NO.** All nodes, edges, and node kinds remain unchanged. The only change is treating a structurally valid model-selected nodeId as a priority signal rather than discarding it entirely. --- ## 60B.6 Walkthrough — What Each Candidate Would Choose Scenario shape: - `n_relocation_decision` (existing unknown): "Which option leaves us better off overall?" - `n_client_retention` (newly-added unknown): "Largest client retention uncertainty" - Model's selectedQuestion.nodeId: `n_relocation_decision` (valid but not material) ### Candidate A — If model had selected n_client_retention Model proposes `selectedQuestion.nodeId = n_client_retention`. Validation passes (exists in addedNodes, kind=unknown, unresolved). Preferred target used. Final nodeId: **`n_client_retention`**. If model instead incorrectly selected `n_relocation_decision`: validation passes (structurally valid). But the fallback path exists — if a diagnostic check flags this as suboptimal (e.g., "parent node has no material connection to continuation reason"), fall back to scoring. The key insight: Candidate A uses the model's choice as the *default*, not an absolute rule, and can fall through to deterministic selection when the preferred target is questionable. **60B.6 final nodeId:** `n_client_retention` (model correctly identifies it in its proposal) ### Candidate B — Deterministic scoring of both candidates Both `n_relocation_decision` and `n_client_retention` enter the scoring pool. No mechanism distinguishes "material" from "incidental". Outcome depends on keyword matching: - `n_relocation_decision` text patterns: "option", "better off", "value" → potential objective(+12) + criteria(+11) = 23 base - `n_client_retention` text patterns: depends on label/description. If it contains "client", "retention", "uncertainty" → partial matches possible **60B.6 final nodeId:** Unpredictable — could be either node depending on exact label/description text. No guarantee of material alignment. ### Candidate C — With continuationTargetNodeId field Model adds `continuationTargetNodeId = n_client_retention` alongside `selectedQuestion.nodeId = n_relocation_decision`. Engine uses the new field as preferred target. Validation passes. Final nodeId: **`n_client_retention`**. But this only works if the model consistently populates the new field. Across providers and runs, consistency is not guaranteed without explicit prompt rules — which defeats the "minimal" requirement. **60B.6 final nodeId:** `n_client_retention` (if populated correctly) --- ## Decision Rule Evaluation ### Criterion 1: Preserves specific material factor identified by reasoning - **A: HIGH** — Model-selected valid node becomes preferred target - **B: MEDIUM** — Unpredictable without domain-specific keyword logic - **C: HIGH** — Explicit field carries the signal ### Criterion 2: Retains deterministic validation/fallback - **A: YES** — Existing validateSelectedQuestion + isSelectableUnresolvedUnknown provide full validation. Fallback at lines 3431-3436 already exists. - **B: PARTIAL** — Scoring heuristics would need new validation - **C: YES** — But requires new validation for the new field ### Criterion 3: Does not rely on recency alone - **A: YES** — Uses explicit model-selected nodeId, not position - **B: NO** — Would need to rely on recency or structural signals (brittle) - **C: YES** — Explicit field, not positional ### Criterion 4: Adds no domain-specific keyword logic - **A: YES** — Purely structural validation - **B: NO** — Requires distinguishing material from incidental via keywords/edges - **C: YES** — New field is structural; but the *prompt rules* for populating it are domain-specific ### Criterion 5: Avoids new schema unless selectedQuestion semantics are genuinely insufficient - **A: YES** — No new schema needed (see semantic honesty analysis above) - **B: YES** — No new schema (but has criterion 4 problem) - **C: NO** — Introduces new field on every proposal shape --- ## Final Choice ### A — USE VALID MODEL-SELECTED NODE AS PREFERRED TARGET #### Why 1. **selectedQuestion.nodeId already carries the correct semantic meaning.** When the model produces `selectedQuestion: {nodeId: "n_client_retention", ...}`, it is explicitly saying "this is the unresolved node we should ask about next." The engine's validation (line 215) and structural checks confirm validity. Ignoring this signal after validation is a semantic contradiction — the field was designed for exactly this purpose, and every validation step confirms the model's choice is structurally sound. 2. **No new schema surface.** Candidate A reuses `selectedQuestion.nodeId` for its existing contract purpose. The only change is giving it priority weight in deterministic selection rather than treating it as advisory-only. This addresses the root cause identified in 60B.7: the field *exists* and *works*, but the engine ignores it. 3. **Fallback is already implemented.** Lines 3418-3437 of apply-proposal.js show the exact fallback pattern: if the selected node disappears after mutation, `selectActiveUnknownCandidate` is called. Candidate A plugs into this existing mechanism at the preference point (before scoring), not by modifying the scorer itself. 4. **Smallest implementation boundary:** - One clarification to prompt Rule 172 ("when model selects a valid unresolved unknown, prefer it as the target") - One modification in `buildSelectedQuestionResult` to honour valid model-selected nodeId before running scoring - Zero new fields, zero new validators, zero new schema 5. **Risk is bounded.** If the model incorrectly selects a non-material node (e.g., parent decision instead of material child), the deterministic fallback at line 3431-3436 can still run when the preferred target becomes unavailable or invalid. The risk of "valid but low-value" selection is already present today — Candidate A does not worsen it; it simply ensures that when the model correctly identifies the material factor, it gets used. ### Smallest implementation boundary 1. **Prompt:** Clarify Rule 172 to indicate preferred-target semantics (one sentence). 2. **apply-proposal.js:** In `buildSelectedQuestionResult`, check if model-selected nodeId is still a valid unresolved unknown in `updatedSituationGraph`; if yes, use it as the selectedNode; if no, fall through to existing `selectActiveUnknownCandidate` path. 3. No schema changes. No new fields. No new scoring dimensions. --- ## Implementation Readiness ### A — READY FOR BOUNDED IMPLEMENTATION One unresolved question for precision: > Should the preferred-target logic apply to *all* valid model-selected nodes, or only when the selected node is in `addedNodes` (i.e., newly created)? > **Answer:** All valid nodes. The constraint is structural validity via existing validation, not origin. This avoids a new heuristic ("newly added = more important") that would violate criterion #4. --- ## Documentation - Created: docs/experiment-60b8.md - Appended to: docs/current-handoff.md - Commit message: experiment: choose material-factor question alignment ## Git status: