112 lines
6.8 KiB
Markdown
112 lines
6.8 KiB
Markdown
# Experiment 60B.77 — Decision Closure Ownership Diagnosis
|
|
|
|
**Date:** 2026-08-15
|
|
**Branch:** `feature/sufficiency-decision-detection-v0.46`
|
|
**Preceded by:** Experiment 60B.76 (premature decision closure observed)
|
|
|
|
## Objective
|
|
|
|
In the single existing 60B.76 run, determine what layer first made `n_product_launch_decision` terminal:
|
|
- A. raw model proposal
|
|
- B. reconciliation synthesis
|
|
- C. propagation-driven parent resolution
|
|
- D. deterministic sufficiency closure gate
|
|
|
|
## Evidence from code inspection (read-only)
|
|
|
|
### Fixture graph structure
|
|
```
|
|
n_product_launch_decision (kind=unknown, status=unknown, childIds=[], parentId=null)
|
|
├── [edges from options via contained_in]
|
|
│
|
|
n_enterprise_customer_signing (kind=unknown, status=unknown, parentId=null)
|
|
└── [edge to opt_launch_this_year via contained_in]
|
|
```
|
|
|
|
The parent has **zero** direct children. The customer is NOT a child of the parent. They are connected via `contained_in` edge from customer to option, not to the decision.
|
|
|
|
### Lifecycle order (applyValidatedProposal)
|
|
|
|
1. Model proposal generation → raw LLM JSON output
|
|
2. Schema validation → parsedProposal.proposal
|
|
3. **Reconciliation** (`reconcileResolutionSemantics`) — adds synthetic entries for nodes in resolvedUnknownNodeIds but missing from updatedNodes
|
|
4. Graph compatibility validation against reconciled proposal
|
|
5. Mutation phase starts — `proposalSnapshot = cloneJsonSafe(validatedProposal)`
|
|
6. Deterministic decomposition runs
|
|
7. **Propagation** (`propagateResolvedChildEvidence`) — walks parentId chain upward for each resolved child unknown
|
|
8. **Deterministic closure** (shouldCloseDecision gate) — checks zero remaining material factors AND explicit confirmation phrase in answer
|
|
9. Active target / selected question selection
|
|
10. Final result returned via orchestrator → harness captures `graphUpdate = proposalSnapshot`
|
|
|
|
### Layer-by-layer analysis for n_product_launch_decision
|
|
|
|
#### 1. RAW MODEL PROPOSAL
|
|
- **Decision present:** UNPROVEN (raw output not captured)
|
|
- **Decision status:** UNPROVEN
|
|
- **Decision in resolvedUnknownNodeIds:** UNPROVEN but likely YES if model closed it
|
|
- **Conclusion:** Cannot determine without raw provider output instrumentation
|
|
|
|
#### 2. RECONCILED PROPOSAL
|
|
- If parent was NOT in model's updatedNodes BUT WAS in resolvedUnknownNodeIds: reconciliation adds a synthetic entry via `buildResolvedUnknownUpdate` (lines 348-352 of apply-proposal.js)
|
|
- If parent WAS in model's updatedNodes: no synthetic entry, entry is raw model data
|
|
- **Critical distinction:** Without capturing raw vs reconciled layers separately, we cannot distinguish these paths
|
|
|
|
#### 3. AFTER PROPAGATION
|
|
**Does propagation resolve the parent? NO.**
|
|
|
|
Code at line 777 of apply-proposal.js:
|
|
```javascript
|
|
if (totalChildren === 0) {
|
|
return { ... nextStatus: parentNode.status, ... }; // unchanged
|
|
}
|
|
```
|
|
|
|
Parent has `childIds: []` → totalChildren = 0 → early return with unchanged status. Propagation does NOT fire.
|
|
|
|
#### 4. AFTER DETERMINISTIC CLOSURE GATE
|
|
**Does deterministic closure fire? NO.**
|
|
|
|
The gate at lines 3808-3814 passes (kind=unknown, not terminal, has contained_in edges). But the actual predicate requires:
|
|
```javascript
|
|
shouldCloseDecision({ decisionNodeId, graph, answer, pendingResolvedIds })
|
|
→ countRemainingMaterialFactors(...) === 0
|
|
&& isUserConfirmationOfNoRemainingUncertainty(answer)
|
|
```
|
|
|
|
`countRemainingMaterialFactors` for n_product_launch_decision:
|
|
- Route A (hierarchy): parent has no children, customer is not a descendant via parentId chain → NO factors
|
|
- Route B (depends_on): no depends_on edges to parent → NO factors
|
|
- Route C (affects→option→decision): no affects/may_cause/causes edges from customer to options → NO factors
|
|
- Route D (containment path): customer ->[contained_in]-> opt_launch → but opt_launch ->[contained_in]-> decision means the containment chain goes option→decision, NOT unknown→option→decision for a material factor. The fromNodeId of each contained_in edge is checked as an unresolved candidate. But opt_launch_this_year is kind=option (not unknown), so it fails isUnresolved().
|
|
|
|
Result: **zero remaining material factors**.
|
|
|
|
But `isUserConfirmationOfNoRemainingUncertainty(answer)` checks the raw answer against bounded CONFIRMATION_PHRASES (no other material uncertainty remains, etc.) and CONFIRMATION_PATTERNS (regex). The 60B.76 answer "confirmed in writing they will not sign if we launch this year" does NOT contain any of these phrases.
|
|
|
|
**Conclusion:** Deterministic closure gate does NOT fire.
|
|
|
|
### Only remaining explanation
|
|
Since propagation does not apply (no children) and deterministic closure does not fire (no confirmation phrase), the **only possible source** of parent resolution is:
|
|
- The model explicitly included the parent in its raw proposal's updatedNodes and/or resolvedUnknownNodeIds
|
|
|
|
## Ownership Conclusion
|
|
|
|
**Choice A — MODEL-LED CLOSURE PROVEN (by elimination)**
|
|
|
|
Neither propagation nor deterministic closure can explain the parent's terminal status. Only the model-led path remains as a viable explanation for both the `updatedNodes` entry and the `resolvedUnknownNodeIds` membership.
|
|
|
|
However, **reconciliation synthesis** cannot be entirely ruled out because:
|
|
- The harness captures `proposalSnapshot` which is a clone of the reconciled proposal
|
|
- If the model put the parent in `resolvedUnknownNodeIds` but NOT in `updatedNodes`, reconciliation would add a synthetic entry
|
|
- Without raw model output logging, this distinction is invisible in existing data
|
|
|
|
**Net assessment:** Model-led closure is the most likely explanation, but exact ownership cannot be definitively proven from the current harness output. The 60B.75 analysis that blamed "deterministic closure" was correct about the symptom (premature parent resolution) but incorrect about the mechanism — it was the model that proposed the closure, not a deterministic gate.
|
|
|
|
## Reasoning Consequence
|
|
|
|
The sufficiency-question logic is NOT at fault for State B not firing in 60B.76. The decision closed via the model's explicit proposal (not via any deterministic or propagation mechanism), and this happened BEFORE question selection could occur because by the time `selectReasoningPattern` runs, the parent is already resolved and there are no remaining unknown active targets.
|
|
|
|
The root cause is: the model determined that resolving the customer factor was sufficient to close the decision, and included both resolutions in its proposal. This means State B (which requires the decision to remain open) cannot be reached when the model closes the decision in the same turn as the factor resolution — regardless of whether explicit sufficiency confirmation was provided.
|
|
|
|
This is a **model behavior / prompt design** issue rather than a deterministic closure guard issue. The prompt may need revision to prevent premature decision closure without explicit sufficiency confirmation.
|