Files

T

robbond e6d0327641 docs: archive historical Confidence Engine evidence

2026-08-19 12:07:17 +01:00

19 KiB

Raw Blame History

Experiment 60B.58 — Decision-Sufficiency Evidence Map

Date: 2026-08-14 Branch: feature/closure-selection-reconciliation-v0.41 Head commit: 2394ad4 experiment: confirm negative closure live

Objective

Answer exactly: what graph evidence already exists in the current architecture that can distinguish "this decision still has a material unresolved factor" from "all represented material uncertainty has been resolved", without relying on option status alone?

No implementation. Read-only analysis of existing topology, code, and fixture.

Method

Analyzed:

Fixture tests/fixtures/pre-anchored-product-launch-customer-signing.json
lib/graph/apply-proposal.js — functions: findDirectChildUnknowns, computeParentProgressState, propagateResolvedChildEvidence, listUnresolvedUnknownCandidates, selectActiveUnknownCandidate, scoreUnknownCandidate, evaluateBranchInteractions, syncParentChildReferences, buildAncestorChain
lib/graph/utils.js — functions: findAffectedNodes, findDependentNodes, scoreUnknownCandidate, countIncomingUnknownDependencies, selectActiveUnknownCandidate, explainUnknownSelection

Checkpoint 1 — Decision-Factor Linkage Map

For n_enterprise_customer_signing in the fixture, here are every structural relationship linking it to the decision and its options:

Relationship: `n_enterprise_customer_signing -> opt_launch_this_year (edge)`

Edge ID: e-customer-signing-to-launch-option
From: n_enterprise_customer_signing (unknown)
To: opt_launch_this_year (option)
Relationship type: contained_in
Direction: unknown → option (upstream dependency flow)
Semantic role: OPTION CONSEQUENCE — the unknown is a condition that affects/attaches to this specific option
Currently used by closure propagation: NO

Relationship: `n_enterprise_customer_signing -> n_product_launch_decision`

Direct edge exists? NO
Direct parentId relationship? NO (both have parentId: null)
Direct depends_on relationship? NO
Direct affects relationship? NO
Semantic role: NONE — structurally disconnected at the decision level
Currently used by closure propagation: NO

Relationship: `opt_launch_this_year -> n_product_launch_decision (edge)`

Edge ID: e-opt-launch-to-dec
Relationship type: contained_in
Direction: option → decision (candidate-for)
Semantic role: CONTAINMENT — option is a candidate for this decision

Relationship: `opt_wait_twelve_months -> n_product_launch_decision (edge)`

Edge ID: e-opt-wait-to-dec
Relationship type: contained_in
Direction: option → decision (candidate-for)
Semantic role: CONTAINMENT — option is a candidate for this decision

Summary of structural attribution:

n_product_launch_decision has no direct unknown child.
  childIds: []
  dependsOn: []
  parentId: null

n_enterprise_customer_signing:
  parentId: null
  dependsOn: []
  affects: []
  
opt_launch_this_year is contained_in n_product_launch_decision.
n_enterprise_customer_signing has an edge to opt_launch_this_year labeled "contained_in".

No propagation path exists from n_enterprise_customer_signing to n_product_launch_decision
through any of: parentId, childIds, depends_on, affects, may_cause, causes, contained_in.

Factor structurally attributable to the decision today: PARTIAL

Checkpoint 2 — Material Unresolved Factor Detection via Existing Topology

parentId / childIds

Classification: INSUFFICIENT for the current case.

findDirectChildUnknowns() (apply-proposal.js:671) finds children where node.parentId === parentNodeId OR edge.fromNodeId -> parentNodeId with relationship=depends_on. In the fixture, no unknown has parentId set to the decision. The factor has parentId: null. Only decomposition-created unknowns get parentId populated (via buildDecompositionContext → buildDecompositionTemplates).

For generic decision-factor relationships created outside decomposition, parentId is not set. The function also scans edges with depends_on from unknown-to-decision, which would catch factor→decision dependencies IF the LLM creates them — but the fixture has no such edge on the factor node.

depends_on

Classification: CONTEXTUAL.

The decision node itself has dependsOn: []. The factor node has dependsOn: []. Neither unknown has a depends_on edge between them in the fixture. If an LLM-created edge connected n_enterprise_customer_signing -> n_product_launch_decision with relationship=depends_on, the existing findDirectChildUnknowns path would catch it (edge scanning at line 678). But this is not present in the fixture.

affects

Classification: INSUFFICIENT.

The factor's affects: [] is empty. No edge originates from the factor with relationship pointing to any decision option beyond the contained_in edge to opt_launch_this_year. The existing code does not use affects for closure propagation — it uses it only for findAffectedNodes (impact scanning, not dependency tracking).

may_cause / causes

Classification: UNUSED BY CURRENT CLOSURE.

These appear in STRUCTURAL_CONSEQUENCE_RELATIONSHIPS at line 1911 of apply-proposal.js but only within the Route B structural context admission check for reasoning-pattern compatibility during new-unknown selection. They are never used in propagateResolvedChildEvidence, computeParentProgressState, or any closure-determining path.

contained_in

Classification: INSUFFICIENT.

The factor has a contained_in edge to opt_launch_this_year. The options have contained_in edges to the decision. However, "contained_in" semantics mean "is-a-candidate-for" in this architecture — it flows from option→decision for containment of candidates. The reverse flow (unknown→option via contained_in) is not interpreted as a dependency. No code traverses contained_in edges in either direction for closure propagation.

direct decision -> unknown edge

Classification: UNUSED BY CURRENT CLOSURE.

No such edges exist in the fixture and none are created by production code for generic factor relationships. Only decomposition children receive depends_on edges to their parent (see line 1637 in apply-proposal.js).

Checkpoint 3 — Remaining-Factor Query

Possible with current schema: PARTIAL
Requires new schema: NO
Existing helper already does this: PARTIAL
Closest existing helper: findDirectChildUnknowns (only catches parentId/depends_on children) + propagateResolvedChildEvidence (only propagates from known children)

Narrowest deterministic predicate derivable from current code:

An unresolved unknown counts against a decision's sufficiency when it either (a) has parentId set to the decision node, or (b) has a depends_on edge pointing to the decision node, or (c) is a new unknown admitted during the same turn through Route A/B structural context embedding.

This predicate is too narrow for the customer-signing case: the factor is linked via option-attachment (contained_in → opt), not parentId or depends_on. No existing function traverses option→decision containment edges to find unknowns that attach to any contained option.

Checkpoint 4 — Self-Counting Problem

Parent decision appears in generic unresolved list: YES

selectActiveUnknownCandidate() (utils.js:593) filters graph.nodes for kind=unknown AND status not in [known, resolved, contradicted] AND id not in resolvedNodeIds. n_product_launch_decision has status=unknown, is not in resolvedNodeIds, so it IS included.

Parent can self-count as remaining unresolved: YES

Because the decision node itself is an unknown with status=unknown, a generic unresolved list will always contain it unless explicitly filtered. After all subordinate factors resolve, the decision node remains in the list — creating exactly the self-counting problem. The system cannot distinguish "the decision itself hasn't been concluded" from "evidence for the decision is incomplete."

Current distinction between decision and factor: PARTIAL

computeParentProgressState() (apply-proposal.js:744) distinguishes parent from children by examining findDirectChildUnknowns(graph, parentNode.id). But this only works when unknowns have parentId/depends_on links to the parent. When a factor is structurally disconnected (as in the fixture), no child-unknown path exists — so there is zero distinction between "parent awaiting conclusion" and "factor beneath parent unresolved."

propagateResolvedChildEvidence() at line 894 filters for node.parentId on resolved children. If parentId is null, nothing propagates upward. The decision node never gets marked "resolved by propagation" when no direct child exists.

Checkpoint 5 — Candidate Assessment

Candidate A — CHILD UNKNOWN COMPLETION ONLY

Close decision only when all direct parentId child unknowns resolve.

Architecture fit: HIGH — uses existing propagateResolvedChildEvidence and computeParentProgressState
Fixes 60B.56: NO — the factor has no parentId to the decision, so completion is never triggered
Premature-closure risk: LOW — requires explicit decomposition relationship
Depends on model compliance: HIGH — only works if LLM always creates parentId links
Requires schema change: YES (for non-decomposition factors) or NO (if we extend parentId semantics)
Principal weakness: Cannot capture generic decision-factor relationships created outside decomposition

Candidate B — RELATIONSHIP-AWARE MATERIAL FACTORS

Close decision when no unresolved decision-relevant unknown remains across approved structural relationships.

Architecture fit: MEDIUM — requires adding traversal of option-attachment edges
Fixes 60B.56: YES — would traverse factor→option(contained_in)→decision path
Premature-closure risk: LOW — only traverses known relationship types
Depends on model compliance: MEDIUM — depends on correct edge creation
Requires schema change: NO (uses existing edge fields)
Principal weakness: Must define which relationships count as "decision-relevant"; currently ambiguous what qualifies

Candidate C — OPTION STATUS

Close when all contained options have status=known.

Architecture fit: HIGH — options already track status
Fixes 60B.56: PARTIAL — addresses symptom but not the causal question
Premature-closure risk: HIGH — option status=known may only mean "the alternative itself is established, not that its comparative value is fully determined"
Depends on model compliance: LOW
Requires schema change: NO
Principal weakness: Premature closure. The fixed answer explicitly states no other uncertainties remain, but the option status alone doesn't prove material evidence is complete

Candidate D — USER DECLARATION ONLY

Close when user explicitly says no other material uncertainty remains.

Architecture fit: MEDIUM — requires capturing and evaluating user statement
Fixes 60B.56: YES — the 60B.56 answer includes "There are no other material uncertainties between launching this year and waiting twelve months."
Premature-closure risk: LOW (with graph guard) / HIGH (without it)
Depends on model compliance: HIGH
Requires schema change: NO
Principal weakness: Relies entirely on model extracting/propagating user statement; no independent graph verification

Candidate E — RELATIONSHIP-AWARE FACTORS + USER DECLARATION

Require both graph evidence of no represented unresolved factor AND explicit user confirmation.

Architecture fit: MEDIUM — combines B and D
Fixes 60B.56: YES — handles both the graph gap and the user statement
Premature-closure risk: LOW — dual-signal requirement reduces false closure
Depends on model compliance: MEDIUM
Requires schema change: NO
Principal weakness: Requires defining "approved structural relationships" for factor-to-decision linkage

Candidate F — MODEL MUST CONTINUE TO OWN CLOSURE

No deterministic propagation beyond existing child mechanism.

Architecture fit: HIGH — current state
Fixes 60B.56: NO — leaves the problem unresolved
Premature-closure risk: NONE (won't close at all)
Depends on model compliance: VERY HIGH
Requires schema change: NO
Principal weakness: The model will keep generating follow-up questions forever for non-decomposition decisions

Checkpoint 6 — Exact 60B.56 Sufficiency Test

Using Candidate E (relationship-aware + user declaration) as the winning model:

Post-60B.56 graph state:

n_enterprise_customer_signing: status=resolved
opt_launch_this_year: status=known, contained_in n_product_launch_decision
opt_wait_twelve_months: status=known, contained_in n_product_launch_decision
n_product_launch_decision: status=unknown, childIds=[], dependsOn=[]

Graph-side check:

Represented unresolved material factors remaining: 0

No unknown has parentId set to the decision. No unknown has depends_on pointing to the decision.
The only structural path from the resolved factor to the decision goes through option-attachment
(factor -> opt_launch_this_year via contained_in edge -> decision via contained_in), which 
isn't traversed by current propagation code. But no UNKNOWN node remains structurally linked 
to any option that belongs to this decision — both options are status=known and contain no 
unresolved unknown children.

However: the factor IS still in the graph as a resolved unknown, not an unknown unknown.
The real question is whether there's an unresolved unknown structurally attached via any 
approved relationship. The answer is NO — all such links would show through existing 
parentId/depends_on routes that are empty.

User statement:

User explicitly says no other material uncertainty remains: YES
"There are no other material uncertainties between launching this year and waiting twelve months."

Would deterministic sufficiency close n_product_launch_decision: CONDITIONAL

The winning rule (Candidate E) would close the decision because:

Graph check passes: no unresolved unknown linked via parentId/depends_on to the decision or its options
User statement provides explicit closure confirmation

Why: The graph-side predicate evaluates empty for this case (no unresolved unknowns in the parentId/depends_on chain). The user statement is captured by the LLM's answer extraction as a "no more uncertainty" signal. Combined, both signals are present.

Counterexample from existing fixture

Testing pre-anchored-decision-options.json where an additional material unknown exists:

If we modify the decision-options fixture to add:

{
  "id": "n_stickiness_uncertainty",
  "label": "Whether engineering retention is achievable",
  "description": "Uncertain whether two senior engineers will remain after relocation, because they account for key delivery capacity.",
  "kind": "unknown",
  "status": "unknown",
  "parentId": null,
  "dependsOn": [],
  "affects": ["opt_relocate"]
}

This unknown has affects pointing to an option contained in the decision. No parentId link exists. The factor would:

Existing counterexample: synthetic extension of pre-anchored-decision-options fixture with n_stickiness_uncertainty having affects → opt_relocate
Remaining material factor: n_stickiness_unclosure (status=unknown)
Would winning rule keep decision open: UNPROVEN — the current graph-side predicate (parentId/depends_on only) would NOT detect this factor. The rule needs the relationship-aware traversal to catch affects→option links.

However, if we extend Candidate E's graph check to include:
- parentId → decision
- depends_on → decision  
- affects → option contained_in decision
Then it WOULD detect n_stickiness_uncertainty and keep the decision open.

Without that extension, both 60B.56 (correct closure) AND this counterexample (incorrect closure) pass through the same predicate — which is exactly the defect we're diagnosing.

CRITICAL DISTINCTION

Choice: E

Why: The analysis identified six candidates for how to determine that a decision's material uncertainty is fully resolved. Candidate E — RELATIONSHIP-AWARE FACTORS + USER DECLARATION — was selected as the winning model because it alone satisfies both requirements simultaneously: (1) graph evidence that no unresolved unknown remains across all structural relationships linking factors to the decision or its options, and (2) explicit user confirmation that nothing else is uncertain. Single-signal approaches (parentId-only, option-status-only, user-declaration-only) each fail on at least one dimension. Candidate E's dual-signal requirement reduces premature-closure risk to LOW. The critical distinction is that closure requires TWO independent signals converging — not one strong signal and not two weak ones. The graph-side signal proves "nothing left unresolved in the model." The user signal proves "nothing left unresolved in reality." Only together do they establish sufficiency.

MINIMUM CORRECTIVE BOUNDARY

Choice: B

Why: The smallest change that makes closure detection correct is extending the unresolved-unknown predicate to traverse option-attachment edges: parentId → decision, depends_on → decision, and affects → option contained_in decision. This is a traversal-extension, not a schema change. No new fields or node types are required. The edge semantics already exist in the graph. Only the propagation logic in propagateResolvedChildEvidence / computeParentProgressState needs to widen its scan to include option-contained unknowns reachable via the approved relationship set. This matches Candidate B from Checkpoint 5.

CLOSURE VS DIRECTION

Can close without preferred option: PARTIAL

Why: Currently, the predicate only checks parentId/depends_on children of the decision node. It does not check unknowns attached to any of the decision's options via affected/contained relationships. Closing would require checking ALL options of the decision for unresolved unknowns, not just those directly under the decision as a child. The architecture supports option-attached factors (as shown by the customer-signing case), but the closure predicate doesn't traverse into them. This is PARTIAL because the infrastructure exists but the traversal gap means only decomposition-child closure works correctly today.

IMPLEMENTATION READINESS

One unresolved question: Which exact relationship types qualify as "decision-relevant" for generic (non-decomposition) factors — affects, may_cause, causes, or all three? 60B.15 established these for context admission but didn't define their closure-weight semantics.

Smallest implementation boundary: Extend propagateResolvedChildEvidence to also scan option-attached unknowns: for each option contained_in the decision, find all unresolved unknowns linked via affects or contained_in edges to that option. Combine with existing parentId/depends_on child scan. If combined result is empty AND user confirmation exists → close decision.

Production code changed: NO

Tests changed: NO

Prompt changed: NO

Schema changed: NO

Ollama calls: 0

Live API calls: 0

Vitest run: NO

Documentation updated: experiment-60b58.md + current-handoff.md

Git status: (to be confirmed after commit)

19 KiB Raw Blame History