confidence-engine/docs/archive/experiments/decision-closure-integration/experiment-60b10.md

Experiment 60B.10 — When should a valid model-selected target override deterministic priority?

Branch: feature/question-target-alignment-v0.27
Date: 2026-08-13
Type: READ-ONLY DESIGN DIAGNOSIS — Resolves the contract conflict between honoring model-selected targets and preserving existing structural overrides.

---

Context

Experiment 60B.9 implemented a blanket "honour model-selected unresolved unknown" rule at line 3680 of apply-proposal.js. This exposed a genuine contract conflict:

NEW desired behaviour: preserve a model-selected material unknown when it is the specific same-turn factor that justifies continuation

EXISTING behaviour (expressed as regression test): deterministic selection may override a valid model-selected unresolved node when another candidate has higher structural/deterministic value

The failing regression: "replaces downstream pricing question with higher-value commercial-value question" proves these behaviours cannot both be preserved if every structurally-valid model target is always preferred.

---

CASE A — 60B.6 material factor

Source: Experiment 60B.6 (docs/current-handoff.md, lines 2703-2742), validated by the reasoning-layer output from live qwen-claude call on pre-anchored-decision-options.json.

Existing decision node:

n_relocation_decision — kind=unknown, status=unknown, label="Which option leaves us better off overall?"
  (pre-existing central decision; activeUnknown before this turn)

opt_relocate — kind=option, label="Relocate to Manchester"

Same-proposal added material unknown:

n_client_retention — kind=unknown, status=unknown
  label: "Largest client retention uncertainty"
  addedEdges: [n_client_retention → opt_relocate, relationship="may_cause"]
  Created because the answer introduced the first new factor that could reverse the preferred option (staying).

Model-selected node: n_client_retention

Desired deterministic target: n_client_retention — because it is the specific material uncertainty whose outcome could change the preferred decision option, justifying continuation. The existing parent (n_relocation_decision) is merely the evaluation context, not the material gap itself.

Graph structure of Case A:

n_relocation_decision (existing unknown)  ←  activeUnknown before proposal
n_build_decision (newly-added state)
n_client_retention (newly-added unknown, may_cause → opt_relocate)
n_relocation_unknown (pre-existing unknown — also unresolved after this turn)

There is NO depends_on edge between n_client_retention and any other newly-added unresolved unknown in this proposal. The may_cause edge connects to an option (non-unknown), not to another unknown node.

---

CASE B — pricing regression

Source: New test added in 60B.9 working tree at tests/graph/apply-proposal.test.js:2006.

Pre-existing model-selected nodes (before proposal):

n_complaint_rate_unknown (kind=unknown, status=unknown) → resolved by this proposal
n_staffing_unknown      (kind=unknown, status=unknown)  → NOT resolved; remains unresolved after this turn

After resolution of n_complaint_rate_unknown: one pre-existing unresolved unknown remains:

n_staffing_unknown

Same-proposal added nodes:

n_commercial_value    — kind=unknown, status=unknown (no dependencies)
n_pricing             — kind=unknown, status=unknown, depends_on=["n_commercial_value"]
n_build_decision      — kind=state (not unknown; irrelevant to selection)

Model-selected nodeId:

n_pricing  (reason: "Model chose a downstream leaf")

Existing deterministic winner:

n_commercial_value (preferred by deterministic scoring over n_pricing because:
  - n_commercial_value has unresolvedParentUnknownCount=0
  - n_pricing has unresolvedParentUnknownCount=1 (depends on n_commercial_value)
  - structural prerequisite relationship: n_commercial_value → depends_on ← n_pricing)

Note: n_staffing_unknown is also an unresolved candidate but its score is lower than both commercial nodes due to keyword matching and downstream count patterns. The test specifically verifies that n_commercial_value wins over the model-selected n_pricing.

Why existing test prefers deterministic winner:

The selected node n_pricing is structurally downstream of another unresolved unknown (n_commercial_value) added in this same proposal. The structural prerequisite chain (commercial value → pricing) means you cannot properly assess n_pricing without first resolving n_commercial_value. Honoring the model's selection of a downstream consequence before its prerequisite understanding would be investigation-order inverted.

Graph structure of Case B:

n_build_decision (newly-added state)
  ├─ n_commercial_value (newly-added unknown, leaf — no upstream unknown dependencies)
  └─ n_pricing        (newly-added unknown, dependent on n_commercial_value via depends_on edge)

---

CANDIDATE A — SAME-PROPOSAL TARGET

Rule: If model-selected nodeId points to an unresolved unknown added in THIS proposal, prefer it as final target. If model-selected nodeId points to a pre-existing unresolved unknown, retain current deterministic selection behaviour.

Assessment:

Criterion	Answer
Materiality fidelity	MEDIUM — Correctly preserves n_client_retention (Case A). But also prefers n_pricing in Case B where the model chose a downstream node over its prerequisite.
Preserves existing pricing regression	NO — In Case B, both n_commercial_value and n_pricing are same-proposal-added. The rule prefers n_pricing (model-selected) over n_commercial_value (structural prerequisite), breaking the regression.
Requires new schema	NO — Uses proposal.addedNodes + selectedQuestion.nodeId, both existing.
Requires new scoring logic	NO — Binary check: isInAddedNodes(selectedNodeId).
Relies on recency alone	YES — "Added in this proposal" is a pure recency signal with no structural or semantic content beyond timing. The model-selected same-proposal node could be upstream prerequisite, downstream consequence, or tangentially-related. All three types would be equally preferred.
Principal risk	Selecting a downstream consequence before its prerequisite understanding. In Case B, this means asking about pricing before defining commercial value — an investigation-order error. Also: any newly-created unknown (material factor OR tangential) gets equal weight when the model explicitly selects it.

Critical flaw for Candidate A:

"Same-proposal-added" encompasses both upstream prerequisites AND downstream consequences. When the model creates a dependency chain (commercial_value → pricing), the rule cannot distinguish which end of the chain is the material uncertainty. It simply picks whichever the model named — which in Case B happens to be the wrong end of the chain.

---

CANDIDATE B — SAME-PROPOSAL + STRUCTURAL RELATION

Rule: Prefer a model-selected same-proposal-added unresolved unknown only when it has no unresolved parent unknowns that were also added in this proposal turn. When such a structural dependency exists, retain deterministic priority over the upstream prerequisite.

Why "unresolved parent unknown from same proposal" is the right structural signal:

When the model creates both an upstream and downstream unknown in the same turn (e.g., commercial_value → pricing), the depends_on edge between them indicates intentional dependency structure — not coincidental timing. The upstream node represents prerequisite understanding; the downstream node represents a consequence of that understanding. Investigation methodology dictates prerequisites before consequences.

When there is NO unresolved parent unknown from the same proposal (as in Case A), the model-selected node is structurally independent within this turn's additions — it has no structural ties to other newly-created unknowns, making it the appropriate material factor target.

Assessment:

Criterion	Answer
Materiality fidelity	HIGH — Case A: n_client_retention has no unresolved parent unknown from same proposal → honored (correct). Case B: n_pricing depends on n_commercial_value (same proposal) → not honored; deterministic selects n_commercial_value (correct).
Preserves existing pricing regression	YES — The structural dependency check prevents honoring n_pricing in Case B.
Existing structure sufficient	YES — Edge relationships (depends_on edges into unknown nodes) and proposal.addedNodes are both pre-existing. No schema changes needed.
New schema required	NO — Uses only existing: proposal.addedNodes, node edge references, isSelectableUnresolvedUnknown.
Principal risk	The structural dependency check could reject a legitimately selected downstream node if the model created a dependency chain for non-investigation-order reasons (e.g., parallel branch creation). However, in practice, depends_on edges between unknown nodes in the same proposal almost always represent intentional prerequisite chains. This is conservative: it errs on the side of addressing prerequisites first.

Implementation boundary (conceptual only):

In apply-proposal.js after line 3680-3695 (existing honor block):

if (validatedProposal.selectedQuestion?.nodeId) {
  const candidateNodeId = validatedProposal.selectedQuestion.nodeId;

  // Check if this candidate is a same-proposal addition
  const addedInThisProposal = validatedProposal.addedNodes.some(
    n => n.id === candidateNodeId
  );

  if (addedInThisProposal && isSelectableUnresolvedUnknown(updatedSituationGraph, candidateNodeId)) {
    // New structural check: does this node have unresolved parent unknowns from same proposal?
    const upstreamParentIds = findUpstreamUnknownParents(candidateNodeId, updatedSituationGraph);
    const parentsAddedThisTurn = upstreamParentIds.filter(
      parentId => validatedProposal.addedNodes.some(n => n.id === parentId && n.kind === "unknown")
    );

    if (parentsAddedThisTurn.length === 0) {
      // No structural dependency on same-turn unknowns → prefer as target
      deterministicSelection = honorModelSelected(...);
    }
    // else: retain deterministic priority (structural prerequisite wins)
  }
}

The findUpstreamUnknownParents function uses existing edge traversal — no schema change.

---

CANDIDATE C — MODEL TARGET SCORING INPUT

Rule: Keep existing deterministic ranking but add a bounded preference/bonus for a valid model-selected node.

Assessment:

Criterion	Answer
Fixes 60B.6 without arbitrary tuning	NO — To fix Case A (where n_client_retention might score below n_relocation_decision), the bonus must be large enough to override typical keyword-scoring gaps (~12-15 points). But in Case B, the same bonus would need to be small enough NOT to override the structural prerequisite preference for n_commercial_value over n_pricing. These are contradictory requirements: the bonus must simultaneously cross a ~10-point gap (Case A) and fail to cross the same ~10-point gap (Case B) without domain-specific knowledge of which gaps are "material" and which are "structural."
Preserves pricing regression	UNKNOWN — Depends on whether the bonus falls below the commercial_value vs pricing score differential. Cannot determine without exact scoring numbers.
Requires numeric weight tuning	YES — Any bounded bonus inherently requires a numeric weight. The question is what value satisfies all cases simultaneously, which cannot be answered without exhaustive regression testing across diverse scenarios.
Semantic honesty	LOW — "Bonus of X points" has no defensible semantic meaning. Why 10? Why 15? There is no principled basis for any specific weight value — it's purely empirical tuning to avoid breaking existing tests. This violates criterion #4 (no domain-specific/heuristic logic).

Critical flaw:

A scoring bonus cannot simultaneously fix Case A and preserve Case B without knowing the score differential between candidates in each case beforehand. This requires tuning that is inherently case-dependent.

---

CANDIDATE D — EXISTING PRIORITY

Rule: Reject preferred model targets entirely; keep current deterministic override for all cases.

Assessment:

Criterion	Answer
Can existing deterministic signals solve 60B.6 generically	NO
Why	The existing deterministic scorer (scoreUnknownCandidate) scores ALL unresolved unknowns by keyword matching + downstream count + unresolved parent penalty. There is NO existing signal for "material uncertainty that justifies continuation." The score for n_client_retention in Case A competes against n_relocation_decision (pre-existing, with accumulated text patterns from the entire decision history). Without materiality metadata, there is no mechanism to distinguish the material gap from the evaluation context.

Why this preserves the existing regression:

Yes — deterministic priority is preserved for ALL cases including Case B. But it also reverts the fix needed for Case A. The material factor identified by the reasoning layer is lost entirely.

---

CRITICAL DISTINCTION

Is "same-proposal-added + explicitly model-selected" a semantically meaningful signal, or merely a recency heuristic in disguise?

Answer: PARTIAL SIGNAL

Why:

What makes it meaningful: When the model creates an unknown node AND selects it as the question target within the same reasoning turn, this carries genuine semantic content: the model's reasoning layer actively discovered this gap and intentionally named it for immediate follow-up. The dual action (creation + selection) signals discovered material uncertainty, not incidental documentation. This is stronger than recency alone because recency could capture any newly-created node regardless of whether it was selected.

What makes it partial: "Same-proposal-added" encompasses three distinct node types:

1. Upstream prerequisites — nodes that other nodes depend on (e.g., commercial_value)
2. Downstream consequences — nodes that depend on other newly-created nodes (e.g., pricing)
3. Tangentially-related nodes — nodes with no dependency relationships to other same-turn nodes (e.g., client_retention in Case A)

The signal is meaningless for distinguishing between types 1, 2, and 3. It treats a prerequisite, a consequence, and an independent material factor identically.

What makes it fully actionable: Combining the model-selection signal with structural analysis of dependency direction:

• Same-proposal-added + model-selected + no upstream unknown dependencies from same proposal = structurally independent material gap → prefer as target
• Same-proposal-added + model-selected + has upstream unknown dependencies from same proposal = downstream consequence in a prerequisite chain → defer to deterministic prerequisite selection

This combination transforms the partial signal into a meaningful investigation-order check, not a recency rule. The structural dependency direction carries semantics about investigation sequence (prerequisites before consequences), which is grounded in established reasoning methodology rather than temporal coincidence.

---

WINNING MODEL

Choice: B — PREFER MODEL-SELECTED SAME-PROPOSAL UNKNOWN ONLY WHEN STRUCTURALLY TIED TO CONTINUED DECISION

Clarified implementation: Prefer model-selected same-proposal-added unresolved unknown when it has no unresolved parent unknowns that were also added in this proposal turn. This is not a broad "structurally tied" requirement — it is specifically a prerequisite-dependency check within the current proposal's scope.

Why:

1. Fixes Case A: n_client_retention has no upstream depends_on edge to any same-turn unknown. Only downstream edges (may_cause → option). No unresolved parent unknown from this turn → preferred as target.

2. Preserves Case B regression: n_pricing has an upstream depends_on edge from n_commercial_value, both added in this proposal → structural dependency prevents honor → deterministic selects n_commercial_value.

3. No domain-specific keywords: Uses only structural edge traversal (existing graph semantics), not text patterns or classification.

4. No new schema: proposal.addedNodes, node edge references, and isSelectableUnresolvedUnknown are all pre-existing.

5. Does NOT make "newest unknown wins" a global rule: Only applies when the model explicitly selects a same-proposal-added node AND it passes the structural independence check. Pre-existing nodes are unaffected. Nodes without explicit model selection are unaffected.

6. Retains deterministic fallback: When the honor-check fails (structural dependency exists) or the preferred target becomes invalid, existing selectActiveUnknownCandidate path is untouched.

Smallest implementation boundary:

• One structural dependency check in the existing honor-model block (lines 3680-3695 of apply-proposal.js)
• Minor clarification to prompt Rule 172 explaining the prerequisite-dependency constraint
• Zero new schema fields, zero new edge types, zero new classification rules

---

IMPLEMENTATION READINESS

A — READY FOR BOUNDED IMPLEMENTATION

One unresolved question for precision:

Should the structural check apply only to depends_on edges, or to any directed edge relationship (e.g., may_cause, affects)? Answer: Only depends_on edges between unknown nodes. may_cause and affects represent consequence relationships in the opposite direction (unknown may cause → option change) and are not prerequisite chains. Investigating whether an unknown may cause something does not require resolving that thing first — only depends_on edges indicate genuine prerequisites.

---

Scope validation

• Question wording/templates: NOT investigated
• Materiality prompt rule: NOT investigated
• Option scoring: NOT investigated
• Utility models: NOT investigated
• Provider behaviour: NOT investigated
• Schema expansion: NOT required
• Recommendation UI: NOT investigated
• Full-suite failures: NOT investigated
• Unrelated orchestrator issues: NOT investigated
• Ollama calls: 0
• Live API calls: 0
• Vitest run: NO

---

Documentation

• Created: docs/experiment-60b10.md
• Appended to: docs/current-handoff.md (below)
• Implementation readiness: A — ready for bounded implementation

---

Git status:

DOCUMENTATION COMMIT BLOCKED BY PARTIAL 60B.9 WORK (4 uncommitted files cannot be cleanly separated from the partial implementation)