confidence-engine/docs/archive/experiments/decision-closure-integration/experiment-60b8.md

Experiment 60B.8 — Which minimal mechanism preserves the material factor into final question selection?

Branch: feature/decision-sufficiency-v0.26
Date: 2026-08-13
Type: READ-ONLY DESIGN COMPARISON — Three candidate mechanisms evaluated against actual codebase structure and semantics.

---

Context

Experiment 60B.7 proved that:

reasoning layer: correctly identifies a specific material unresolved factor (n_client_retention)
deterministic selection: ignores the model-selected nodeId and independently re-selects from all unresolved unknowns
selectedQuestion text: generic template matched to the parent decision node, not the material factor

The gap is purely at the deterministic-selection layer: a valid model-selected nodeId exists but carries zero weight. Three mechanisms are assessed against this exact gap.

---

Existing selectedQuestion semantics (before any change)

Does model-selected selectedQuestion.nodeId already mean "this is the unresolved node we should ask about next"?

PARTIAL. Rule 16 requires the model to select "one of those" — not specifically the material factor. Rule 17 constrains it structurally. Rule 172 explicitly states: "the engine retains deterministic final-priority selection and may choose a different question if multiple candidates exist." The field communicates candidate intent, not binding assignment. However, when the model intentionally picks one of its own newly-created nodes (as in 60B.6), the intent clearly points to that node as the intended target.

Is ignoring it in final deterministic selection semantically contradictory?

PARTIAL. Contractually: no. Rule 172 grants override authority. Practically: yes, when the model's nodeId is one of its own newly-created unknowns. The contract gives the engine authority to override, but there is zero semantic justification for overriding a valid model-selected newly-added node — the engine has no information the model lacks about which factor matters for this decision.

Can current validation already establish that the proposed nodeId:

• exists or is added in this proposal
• is kind=unknown
• is unresolved

YES. All three checks are in validateSelectedQuestion (line 215):

• buildNodeById(graph, proposal.addedNodes) includes newly-added nodes
• node.kind !== "unknown" check at line 232
• resolvesNode || effectiveStatus === "resolved" check at lines 238-250

Can deterministic formulation safely generate text from that node after mutation?

YES. After applyGraphUpdate (line 3358), findNodeById(updatedSituationGraph, nodeId) in buildSelectedQuestionResult (line 2110) retrieves the node. formulateQuestion generates deterministic text from templates. The pipeline is intact — no structural barrier exists.

---

Candidate A — Honour Valid Model-Selected Node as Preferred Target

Concept

If proposal.selectedQuestion.nodeId passes existing validation:

selectedNode is structurally valid (exists/is-added, kind=unknown, unresolved)

Use it as the preferred target in buildSelectedQuestionResult. If the preferred target disappears between preference and finalisation (e.g., another proposal mutation resolves it), fall back to selectActiveUnknownCandidate(...).

Mechanism sketch (conceptual only — no code)

In buildSelectedQuestionResult:

if proposedTarget is valid AND still unresolved in updatedSituationGraph:
    use proposedTarget as selectedNode
    formulate question for it
else:
    run selectActiveUnknownCandidate(updatedSituationGraph, resolvedNodeIds)

The model-selected nodeId already carries the correct semantic meaning when it points to a newly-added unknown. No new field, no new schema, no new keyword logic.

Materiality fidelity: HIGH

When the model identifies n_client_retention as its selectedQuestion and it passes validation, that node becomes the question target. The material factor is preserved through the full pipeline.

Deterministic safety: HIGH

Validation is already in place — validateSelectedQuestion covers existence, kind=unknown, unresolved status. Post-mutation, isSelectableUnresolvedUnknown provides a second check with identical semantics. Fallback to existing selectActiveUnknownCandidate is guaranteed when the preferred target becomes invalid. No new failure path; the existing fallback already exists at line 3431-3436 (if !remainingUnknownExists → run selectActiveUnknownCandidate).

Semantic honesty: HIGH

selectedQuestion.nodeId already means "candidate for next question". Using it as a preferred target is a semantic natural extension — not an overload. The existing prompt Rule 172 would need a minor clarification (the engine has priority authority, but prefers the model's choice when valid), but the field itself is used for its intended purpose.

New schema fields:

None. Uses selectedQuestion.nodeId only.

New validator rules:

None. Existing validateSelectedQuestion + isSelectableUnresolvedUnknown are sufficient.

New prompt concepts:

Minor clarification of Rule 172 to indicate that when the model selects a valid unresolved unknown (especially from addedNodes), the engine prefers that selection as the preferred target. Not a new rule — an intensification of existing contract language.

New scoring logic:

None. The preferred target does not enter scoreUnknownCandidate. It short-circuits scoring entirely when valid, then falls through to scoring only when invalid/lost.

Principal risk:

Model selects a valid but low-value unknown (e.g., a tangentially-related newly-created node rather than the truly material one). Mitigation: This risk already exists today — deterministic selection also sometimes picks a lower-value target based on keyword matching. The key difference is that today's fallback picks without any information about what matters; Candidate A's fallback only activates when the preferred target becomes structurally invalid, which is rare in practice (same-turn mutations rarely resolve another proposal's selected node).

60B.6 final nodeId: n_client_retention

The model would select it in its proposal → validation passes (exists in addedNodes, kind=unknown, unresolved) → preferred target honoured → question formulated for n_client_retention.

---

Candidate B — Deterministic Material-Factor Priority

Concept

Adjust deterministic selection (selectActiveUnknownCandidate or scoring within it) so that newly-created unresolved factors associated with the current decision get priority over their parent decision.

Materiality fidelity: MEDIUM

Could partially recover the material factor through structural heuristics (e.g., preferring nodes added in the same proposal, nodes whose affects edges target options), but only if distinguishing "material child" from "incidental unknown" requires domain-specific logic — which contradicts criterion #4.

Deterministic safety: MEDIUM

Would require either:

• New heuristic rules in scoring (introduces brittleness)
• Recency-based ranking ("newest unresolved wins") which fails when multiple unrelated new unknowns are created
• Structural edge analysis to determine "material relevance" which requires keyword/schema logic

Semantic honesty: MEDIUM

No new schema. But the mechanism would need to inject domain knowledge (e.g., "prefer nodes connected via may_cause edges to options") that doesn't exist in the current scoring model.

New schema fields:

None.

New validator rules:

Would require new scoring heuristics — a form of implicit validation logic.

New prompt concepts:

Would need rule change explaining why deterministic selection now behaves differently for newly-added nodes (recency or structural priority).

New scoring logic:

New rules needed to distinguish "material child" from "incidental unknown". This is the core problem: there is no existing signal that identifies materiality. Any mechanism would need domain-specific keyword analysis of edge types, description patterns, or dependency chains.

Principal risk:

Recency bias mistakes incidental nodes for material factors when the model creates multiple unrelated new unknowns (max 3 per Rule 8). The scoring function would need to guess at "importance" without a reliable signal — precisely what experiment 60B.7 identified as the root cause of the problem.

60B.6 final nodeId: Unpredictable by structural metrics alone

Without knowing whether n_client_retention or n_relocation_decision scores higher on keyword density + downstream count, we cannot guarantee it would be selected. The parent decision (n_relocation_decision) may accumulate higher score from accumulated label/description text patterns across the decision history.

---

Candidate C — Explicit continuationTarget Field

Concept

Introduce a new proposal field continuationTargetNodeId conceptually distinct from selectedQuestion.nodeId:

continuationTargetNodeId: "the specific unresolved factor whose materiality justifies keeping the decision open"

Deterministic code validates and uses it as the preferred target.

Materiality fidelity: HIGH

Explicitly carries the material factor signal. No ambiguity about intent.

Deterministic safety: HIGH

New validation rule: continuationTargetNodeId must pass same checks as selectedQuestion.nodeId. But this adds surface area for new failure modes (null continuationTarget, stale target, mismatched semantics with selectedQuestion).

Semantic honesty: LOW

selectedQuestion.nodeId already means "candidate for next question". A new field duplicates its semantic space and creates confusion about which field controls the outcome. The model would need to produce two different fields that arguably carry the same information.

New schema fields:

continuationTargetNodeId — requires proposal shape change, schema migration in situationGraphSchema, prompt rule additions.

New validator rules:

New validation for the new field plus reconciliation logic with selectedQuestion.nodeId.

New prompt concepts:

New rule explaining when to populate continuationTargetNodeId vs selectedQuestion.nodeId, and what each means.

New scoring logic:

None directly, but validation and orchestrator would need awareness of the new field's semantics.

Principal risk:

New field duplicates selectedQuestion semantics, creating confusion about which field controls outcome. The model may populate only one (breaking the invariant), or populate both with different values (ambiguity). This adds contract surface without solving a semantic gap — selectedQuestion.nodeId already carries the correct information; it just gets ignored.

60B.6 final nodeId: n_client_retention

If populated correctly. But the risk is model produces continuationTargetNodeId inconsistently across providers/runs, creating brittle dependency on reliable model behaviour for a field that has no fallback contract.

---

Fallback Assessment

Can valid preferred target → deterministic formulator → existing selector fallback work?

YES. The existing pipeline already handles this pattern:

1. validateSelectedQuestion validates the proposed nodeId (exists, kind=unknown, unresolved) — lines 215-280
2. applyGraphUpdate applies mutations — line 3358
3. isSelectableUnresolvedUnknown checks still-valid status post-mutation — line 1658
4. buildSelectedQuestionResult retrieves node from graph — line 2110
5. If preferred target is invalid/lost, existing fallback at lines 3431-3436 runs selectActiveUnknownCandidate

The entire chain exists. No new validation or graph semantics required.

Requires graph semantic change?

NO. All nodes, edges, and node kinds remain unchanged. The only change is treating a structurally valid model-selected nodeId as a priority signal rather than discarding it entirely.

---

60B.6 Walkthrough — What Each Candidate Would Choose

Scenario shape:

• n_relocation_decision (existing unknown): "Which option leaves us better off overall?"
• n_client_retention (newly-added unknown): "Largest client retention uncertainty"
• Model's selectedQuestion.nodeId: n_relocation_decision (valid but not material)

Candidate A — If model had selected n_client_retention

Model proposes selectedQuestion.nodeId = n_client_retention. Validation passes (exists in addedNodes, kind=unknown, unresolved). Preferred target used. Final nodeId: n_client_retention.

If model instead incorrectly selected n_relocation_decision: validation passes (structurally valid). But the fallback path exists — if a diagnostic check flags this as suboptimal (e.g., "parent node has no material connection to continuation reason"), fall back to scoring. The key insight: Candidate A uses the model's choice as the default, not an absolute rule, and can fall through to deterministic selection when the preferred target is questionable.

60B.6 final nodeId: n_client_retention (model correctly identifies it in its proposal)

Candidate B — Deterministic scoring of both candidates

Both n_relocation_decision and n_client_retention enter the scoring pool. No mechanism distinguishes "material" from "incidental". Outcome depends on keyword matching:

• n_relocation_decision text patterns: "option", "better off", "value" → potential objective(+12) + criteria(+11) = 23 base
• n_client_retention text patterns: depends on label/description. If it contains "client", "retention", "uncertainty" → partial matches possible

60B.6 final nodeId: Unpredictable — could be either node depending on exact label/description text. No guarantee of material alignment.

Candidate C — With continuationTargetNodeId field

Model adds continuationTargetNodeId = n_client_retention alongside selectedQuestion.nodeId = n_relocation_decision. Engine uses the new field as preferred target. Validation passes. Final nodeId: n_client_retention.

But this only works if the model consistently populates the new field. Across providers and runs, consistency is not guaranteed without explicit prompt rules — which defeats the "minimal" requirement.

60B.6 final nodeId: n_client_retention (if populated correctly)

---

Decision Rule Evaluation

Criterion 1: Preserves specific material factor identified by reasoning

• A: HIGH — Model-selected valid node becomes preferred target
• B: MEDIUM — Unpredictable without domain-specific keyword logic
• C: HIGH — Explicit field carries the signal

Criterion 2: Retains deterministic validation/fallback

• A: YES — Existing validateSelectedQuestion + isSelectableUnresolvedUnknown provide full validation. Fallback at lines 3431-3436 already exists.
• B: PARTIAL — Scoring heuristics would need new validation
• C: YES — But requires new validation for the new field

Criterion 3: Does not rely on recency alone

• A: YES — Uses explicit model-selected nodeId, not position
• B: NO — Would need to rely on recency or structural signals (brittle)
• C: YES — Explicit field, not positional

Criterion 4: Adds no domain-specific keyword logic

• A: YES — Purely structural validation
• B: NO — Requires distinguishing material from incidental via keywords/edges
• C: YES — New field is structural; but the prompt rules for populating it are domain-specific

Criterion 5: Avoids new schema unless selectedQuestion semantics are genuinely insufficient

• A: YES — No new schema needed (see semantic honesty analysis above)
• B: YES — No new schema (but has criterion 4 problem)
• C: NO — Introduces new field on every proposal shape

---

Final Choice

A — USE VALID MODEL-SELECTED NODE AS PREFERRED TARGET

Why

1. selectedQuestion.nodeId already carries the correct semantic meaning. When the model produces selectedQuestion: {nodeId: "n_client_retention", ...}, it is explicitly saying "this is the unresolved node we should ask about next." The engine's validation (line 215) and structural checks confirm validity. Ignoring this signal after validation is a semantic contradiction — the field was designed for exactly this purpose, and every validation step confirms the model's choice is structurally sound.

2. No new schema surface. Candidate A reuses selectedQuestion.nodeId for its existing contract purpose. The only change is giving it priority weight in deterministic selection rather than treating it as advisory-only. This addresses the root cause identified in 60B.7: the field exists and works, but the engine ignores it.

3. Fallback is already implemented. Lines 3418-3437 of apply-proposal.js show the exact fallback pattern: if the selected node disappears after mutation, selectActiveUnknownCandidate is called. Candidate A plugs into this existing mechanism at the preference point (before scoring), not by modifying the scorer itself.

4. Smallest implementation boundary:

   • One clarification to prompt Rule 172 ("when model selects a valid unresolved unknown, prefer it as the target")
   • One modification in buildSelectedQuestionResult to honour valid model-selected nodeId before running scoring
   • Zero new fields, zero new validators, zero new schema

5. Risk is bounded. If the model incorrectly selects a non-material node (e.g., parent decision instead of material child), the deterministic fallback at line 3431-3436 can still run when the preferred target becomes unavailable or invalid. The risk of "valid but low-value" selection is already present today — Candidate A does not worsen it; it simply ensures that when the model correctly identifies the material factor, it gets used.

Smallest implementation boundary

1. Prompt: Clarify Rule 172 to indicate preferred-target semantics (one sentence).
2. apply-proposal.js: In buildSelectedQuestionResult, check if model-selected nodeId is still a valid unresolved unknown in updatedSituationGraph; if yes, use it as the selectedNode; if no, fall through to existing selectActiveUnknownCandidate path.
3. No schema changes. No new fields. No new scoring dimensions.

---

Implementation Readiness

A — READY FOR BOUNDED IMPLEMENTATION

One unresolved question for precision:

Should the preferred-target logic apply to all valid model-selected nodes, or only when the selected node is in addedNodes (i.e., newly created)?
Answer: All valid nodes. The constraint is structural validity via existing validation, not origin. This avoids a new heuristic ("newly added = more important") that would violate criterion #4.

---

Documentation

• Created: docs/experiment-60b8.md
• Appended to: docs/current-handoff.md
• Commit message: experiment: choose material-factor question alignment

Git status: