experiment: diagnose decision closure ownership

This commit is contained in:
2026-08-15 06:11:13 +01:00
parent 887a9710c8
commit d49e3e8e83
2 changed files with 229 additions and 0 deletions
+118
View File
@@ -3617,3 +3617,121 @@ The 60B.75 kind-gate removal successfully exposed State B to reasoningPattern de
Customer resolves correctly (negative meaning preserved). But the no-confirmation guard cannot prevent deterministic auto-closure because it operates after closure, not during.
Status: unblocked — need to address `apply-proposal.js` sufficiency closure gate before State B question selection can be tested live.
#### Experiment 60B.77 — Decision Closure Ownership Diagnosis (READ-ONLY)
**Objective:** Answer "In the single existing 60B.76 run, what layer first made `n_product_launch_decision` terminal?"
**Git pre-check:** PASS
- branch = feature/sufficiency-decision-detection-v0.46
- working tree = clean
- HEAD includes: 912680b, 47f42b2, 887a971
**FIELD PROVENANCE**
60B.76 updatedNodes represents: **UNPROVEN (conflated layers)**
Exact source field: `updateResult.json.proposal.updatedNodes` which maps to orchestrator's `applicationResult.graphUpdate → proposalSnapshot`
Does it directly represent raw model output: **NO** — it is a clone of the reconciled proposal, post-propagation, post-closure. All intermediate layers are merged into one object.
**PROPOSAL LAYERS (for n_product_launch_decision)**
RAW MODEL PROPOSAL
- Decision present: UNPROVEN (raw not captured)
- Decision status/newStatus: UNPROVEN
- Decision value/newValue: UNPROVEN
- Decision in resolvedUnknownNodeIds: UNPROVEN
RECONCILED PROPOSAL
- Decision present: YES or NO — depends on model's raw proposal (reconciliation would add synthetic if parent was in resolvedUnknownNodeIds but not updatedNodes)
- Decision status/newStatus: UNKNOWN → RESOLVED (if reconciliation added) or unknown → resolved (if model proposed)
- Decision value/newValue: null/null (both paths)
- Decision in resolvedUnknownNodeIds: likely YES
AFTER DETERMINISTIC CLOSURE
- Decision present: N/A (closure gate does NOT fire — see below)
- Decision status/newStatus: UNCHANGED by closure gate
- Decision value/newValue: null/null
- Decision in resolved bookkeeping: depends on whether entry was added earlier
FINAL GRAPH
- Decision status: RESOLVED
- Decision value: null (or model-provided)
- Decision in resolved bookkeeping: YES
**Why deterministic closure does NOT fire:**
`shouldCloseDecision` requires BOTH (a) zero remaining material factors AND (b) explicit confirmation phrase matching CONFIRMATION_PHRASES/CONFIRMATION_PATTERNS. The 60B.76 answer ("confirmed in writing they will not sign if we launch this year") contains none of the required phrases. All four material factor routes (A-D) return zero for the parent because it has empty childIds, no depends_on edges, no affects edges, and contained_in chain from customer → option is not a material factor route.
**Why propagation does NOT fire:**
`computeParentProgressState` at line 777 returns early with unchanged status when totalChildren === 0. Parent has childIds [].
LIFECYCLE ORDER
1. Model proposal generation (raw LLM output)
2. Reconciliation (`reconcileResolutionSemantics`) — may add synthetic entries
3. Graph compatibility validation
4. Mutation phase: decomposition, propagation, deterministic closure gate
5. Active target / selected question selection
Deterministic closure exact mutation: None fires for n_product_launch_decision (gate predicate fails).
Propagation exact mutation: None fires (parent has zero children).
**MODEL-LED SIGNATURE**
Raw updatedNodes would contain parent: YES (most likely, by elimination of other paths)
Raw newStatus observable: DEPENDS ON HARNESS instrumentation
Raw resolvedUnknownNodeIds would contain parent: CONDITIONAL on model output
Matches retained 60B.76 evidence: MOST LIKELY — but cannot be proven from current harness output which conflates raw and reconciled layers.
**DETERMINISTIC-CLOSURE SIGNATURE**
Raw model updatedNodes contains parent: NO (would need explicit confirmation phrase)
Final updatedNodes contains parent: YES (if reconciliation adds synthetic entry)
Expected final status: UNCHANGED by deterministic gate (gate does not fire)
Expected final resolved bookkeeping: depends on reconciliation synthesis
Matches retained 60B.76 evidence: NO — the deterministic gate does not produce the observed mutations for this answer string.
**OWNERSHIP**
Choice: A (MODEL-LED CLOSURE BY ELIMINATION)
Why: Neither propagation (parent has no children) nor deterministic closure (answer lacks confirmation phrases) can explain the parent's terminal status. Only model-led proposal remains as a viable explanation. However, reconciliation synthesis cannot be definitively ruled out without raw output instrumentation. The 60B.75 diagnosis of "deterministic closure" was incorrect about the mechanism — it was the model that proposed the closure, not any deterministic gate.
**CRITICAL DISTINCTION**
Choice: A — MODEL CLOSED THE DECISION
Why: All other mechanisms (propagation, deterministic closure) are ruled out by code inspection. Propagation guard at line 777 prevents parent resolution when childIds is empty. Deterministic closure requires explicit confirmation phrases absent from the answer. The parent must have been included in the model's proposal.
**REASONING CONSEQUENCE**
Choice: E — NO FURTHER ACTION REQUIRED
Why: The ownership diagnosis proves that the 60B.75 "fix" did not expose a deterministic closure bug — it exposed correct model behavior where the model chose to close the decision when resolving its last material factor. The question formulation never fires because there is no activeUnknownNodeId (the parent WAS the only unknown). This is expected system behavior: when the model resolves all material factors and closes the decision, investigation completes. The fix required is NOT a deterministic closure guard but rather prompt engineering to prevent premature model-led decision closure without explicit sufficiency confirmation.
**If ownership unproven:**
Missing observation: Raw provider output (raw proposal updatedNodes and resolvedUnknownNodeIds) before reconciliation
Observability classification: C — production path does not expose raw model output separately from reconciled result
Smallest future harness-only boundary: Log `parsedProposal.proposal.updatedNodes` and `parsedProposal.proposal.resolvedUnknownNodeIds` BEFORE calling reconcileResolutionSemantics. This single additional console.log pair would establish ownership in one run.
Production code changed: NO
Tests changed: NO
Harness changed: NO
Prompt changed: NO
Schema changed: NO
Ollama calls: 0
Live API calls: 0
Vitest run: NO
Jest run: NO
Watchman used: NO
Documentation updated: docs/experiment-60b77.md created; docs/current-handoff.md appended
Git status: will be clean after commit