experiment: locate post-mutation question guard

This commit is contained in:
2026-08-14 08:38:23 +01:00
parent 88a80180b7
commit 5c6b3421dd
2 changed files with 368 additions and 0 deletions
+3
View File
@@ -2961,3 +2961,6 @@ Experiment 60B.37 ran the customer-signing followup fixture through a single bou
--- ---
Experiment 60B.38 was a read-only diagnosis of why `n_product_launch_decision` becomes known in the same update yet still survives as the final selectedQuestion target. **Classification: B — QUESTION TARGET VALIDATION IS WRONG.** Two independent gaps discovered in the selectable-node predicate chain: (1) `validateSelectedQuestion` at lib/graph/apply-proposal.js:246 checks `effectiveStatus === "resolved"` only, NOT `"known"`; (2) `isSelectableUnresolvedUnknown` at lib/graph/apply-proposal.js:1663 excludes ["resolved", "contradicted"] but NOT "known". These gaps propagate through the entire pipeline: `selectActiveUnknownCandidate` in utils.js has no status filter at all. The decision node transitioned via `updatedNodes.newStatus="known"` rather than `resolvedUnknownNodeIds`, and no predicate catches this gap. **Minimum corrective boundary:** C — unify all final targets through one canonical post-mutation unresolved/selectable check. Add "known" to the exclusion list in `isSelectableUnresolvedUnknown` and add `effectiveStatus === "known"` check in `validateSelectedQuestion`. No production code changed. 0 Ollama calls. Pure code inspection. Full trace in docs/experiment-60b38.md. Experiment 60B.38 was a read-only diagnosis of why `n_product_launch_decision` becomes known in the same update yet still survives as the final selectedQuestion target. **Classification: B — QUESTION TARGET VALIDATION IS WRONG.** Two independent gaps discovered in the selectable-node predicate chain: (1) `validateSelectedQuestion` at lib/graph/apply-proposal.js:246 checks `effectiveStatus === "resolved"` only, NOT `"known"`; (2) `isSelectableUnresolvedUnknown` at lib/graph/apply-proposal.js:1663 excludes ["resolved", "contradicted"] but NOT "known". These gaps propagate through the entire pipeline: `selectActiveUnknownCandidate` in utils.js has no status filter at all. The decision node transitioned via `updatedNodes.newStatus="known"` rather than `resolvedUnknownNodeIds`, and no predicate catches this gap. **Minimum corrective boundary:** C — unify all final targets through one canonical post-mutation unresolved/selectable check. Add "known" to the exclusion list in `isSelectableUnresolvedUnknown` and add `effectiveStatus === "known"` check in `validateSelectedQuestion`. No production code changed. 0 Ollama calls. Pure code inspection. Full trace in docs/experiment-60b38.md.
---
Experiment 60B.40 performed read-only post-mutation guard location diagnosis for same-turn terminal-target survival (known status). **Classification: B — PREFERRED TARGET NEEDS EXPLICIT POST-MUTATION REVALIDATION.** Traced all five post-mutation sources that can supply the final selectedQuestion node after applyGraphUpdate (line 3671): Source A (selectActiveUnknownCandidate, line 3722) has zero status filtering; Source B (isSelectableUnresolvedUnknown preservation at line 3764) excludes ["resolved", "contradicted"] but not "known"; Source C (model-selection honour at line 3984) has same gap; Source D (remainingUnknownExists inline check at line 3710) checks only kind + resolvedNodeIds, no status; Source E/F (selectPatternCompatibleUnknownCandidate and listUnresolvedUnknownCandidates) share the same ["resolved", "contradicted"] exclusion gap. Earliest safe guard point: line 3704 — updatedSituationGraph exists, proposal accepted, selected target can still be discarded, question not yet finalized. Fallback behaviour confirmed as C (both A+B): selectAnotherUnresolvedCandidate when one exists, return null when none remain. Minimum corrective boundary: E (minimum combination) — add "known" to isSelectableUnresolvedUnknown exclusion list AND add known exclusion to remainingUnknownExists inline check at line 3710-3711. Leaves residual gap in selectActiveUnknownCandidate if all candidates cascade to known status. One unresolved question requires clarification before readiness. Full analysis in docs/experiment-60b40.md. No production code changed. 0 Ollama calls. Pure code inspection.
+365
View File
@@ -0,0 +1,365 @@
# Experiment 60B.40 — Locate post-mutation question guard
**Date:** 2026-08-14
**Branch:** `feature/known-target-exclusion-v0.36`
**Objective:** Identify the smallest post-mutation guard that can discard a selected target which became terminal in the same update, without invalidating the proposal or disturbing valid unresolved preferred-target behaviour.
## Checkpoint 1 — Post-mutation target sources
After `applyGraphUpdate(...)` (line 3671), five independent sources can supply the eventual final `selectedQuestion` node:
### Source A — deterministicSelection via selectActiveUnknownCandidate (line 3722)
```
function/location:
applyValidatedProposal line 3722 → selectActiveUnknownCandidate(lib/graph/utils.js:593)
uses updated graph:
YES — passes updatedSituationGraph (built at line 3680)
passes through isSelectableUnresolvedUnknown:
NO — direct call, no intermediate filtering
can select status=known today:
YES — selectActiveUnknownCandidate checks only node.kind === "unknown" && !resolvedNodeIds.includes(n.id). Zero status filtering.
```
### Source B — preservedSelectedChildNode via isSelectableUnresolvedUnknown (line 3764)
```
function/location:
applyValidatedProposal line 3764 → isSelectableUnresolvedUnknown(updatedSituationGraph, selectedChildNodeId)
uses updated graph:
YES — updatedSituationGraph
passes through isSelectableUnresolvedUnknown:
YES (is itself the call)
can select status=known today:
YES — isSelectableUnresolvedUnknown excludes ["resolved", "contradicted"] only. "known" slips through.
```
If Source B passes, deterministicSelection gets set to the terminal node at line 3787-392. This becomes the final selectedQuestion via line 4155/4063 → effectiveSelectedQuestion → line 4377.
### Source C — model-selection honour path (line 3976)
```
function/location:
applyValidatedProposal lines 3976-3998 (the "model-selection honour" block)
uses updated graph:
YES — isSelectableUnresolvedUnknown(updatedSituationGraph, candidateNodeId)
passes through isSelectableUnresolvedUnknown:
YES (line 3984)
can select status=known today:
YES — the gap at line 1663 lets known pass. However, this path also requires candidateWasAddedThisProposal (line 3979-3981), so it only affects newly-added nodes, not pre-existing ones like in 60B.37/38/40.
```
### Source D — remainingUnknownExists guard (line 3705)
```
function/location:
applyValidatedProposal lines 3705-3712
uses updated graph:
YES — checks against updatedSituationGraph.nodes
passes through isSelectableUnresolvedUnknown:
NO — inline .some() check, no reuse of any predicate function
can select status=known today:
YES — checks node.kind === "unknown" && !resolvedNodeIds.includes(node.id). Zero status filtering. This source is what keeps the known node alive as newActiveUnknownNodeId when proposal.selectedQuestion.nodeId exists.
```
### Source E — carriedActiveUnknownStillUnresolved (line 3847)
```
function/location:
applyValidatedProposal lines 3844-3854
uses updated graph:
YES — findNodeById(updatedSituationGraph, ...) and updatedSituationGraph.resolvedNodeIds
passes through isSelectableUnresolvedUnknown:
NO — inline check with same ["resolved", "contradicted"] gap
can select status=known today:
YES — same pattern as Source D: kind + resolvedNodeIds only.
```
### Source F — selectPatternCompatibleUnknownCandidate (line 2169)
```
function/location:
lib/graph/apply-proposal.js line 2169, used at line 3809
uses updated graph:
YES — passed as graph parameter
passes through isSelectableUnresolvedUnknown:
NO — its own inline filter at line 2186 has the same ["resolved", "contradicted"] gap.
can select status=known today:
YES
```
### Source G — listUnresolvedUnknownCandidates / listEligibleUnknownCandidates (lines 1668-1692)
```
function/location:
lib/graph/apply-proposal.js lines 1668-1681 and 1683-1692
uses updated graph:
YES — passed as first parameter
passes through isSelectableUnresolvedUnknown:
NO — independent filter with identical gap (line 1677).
can select status=known today:
YES
```
---
## Checkpoint 2 — Earliest safe post-mutation boundary
**Function:** `applyValidatedProposal` in lib/graph/apply-proposal.js
**Approximate location:** Between line 3680 (updatedSituationGraph construction) and line 3701 (proposal.selectedQuestion.nodeId → newActiveUnknownNodeId assignment).
More precisely: the optimal insertion point is at **line 3704**, right after the block that sets newActiveUnknownNodeId from proposal.target but before the remainingUnknownExists check at line 3705.
**Updated graph available:** YES — `updatedSituationGraph` exists with correct post-mutation node statuses including all same-turn transitions (e.g., unknown → known).
**Proposal already accepted:** YES — proposal compatibility passed at line 3612, structural admission complete, applyGraphUpdate succeeded at line 3671. The proposal is committed.
**Fallback still possible:** YES — if we add a status check to remainingUnknownExists at line 3705-3712, it returns false for known-status nodes, which triggers the fallback path at line 3714-3720 (selectActiveUnknownCandidate or null). Similarly, adding known exclusion to isSelectableUnresolvedUnknown would cause Source B/C to fail and trigger reselection.
**Question text not yet finalized:** YES — deterministicSelection is built after this point (line 3722), finalSelectedQuestion at line 4055, effectiveSelectedQuestion at line 4147, selectedQuestion output at line 4377. All of these occur after the guard point.
**Inputs available:**
- `updatedSituationGraph` — fully post-mutation graph with all status transitions visible
- `validatedProposal.selectedQuestion.nodeId` — the proposal's target
- `deterministicSelection` — candidate for replacement (set at line 3722 or later)
- `eligibleCandidates` — list of eligible unresolved candidates (built at lines 3894-3911)
**Output controlled:**
- `newActiveUnknownNodeId` — set at lines 3696-3703, corrected at line 4001-4024 based on deterministicSelection
- `deterministicSelection` — set at line 3722/3742/3787/3987 and used to build the final question
---
## Checkpoint 3 — Fallback behaviour
If a proposal-selected target becomes terminal (known/resolved) post-mutation, existing code already provides fallback:
### Choice: C — BOTH A AND B
**Exact path for A (fallback to another candidate):**
When remainingUnknownExists at line 3705 returns false (because the known node is correctly excluded), or when isSelectableUnresolvedUnknown at line 3764 rejects it, the flow falls through:
- Line 3714-3720: `selectActiveUnknownCandidate(updatedSituationGraph, updatedSituationGraph.resolvedNodeIds)` picks the highest-score unresolved unknown.
- If that returns null (no candidates), newActiveUnknownNodeId becomes null at line 3719.
**Exact path for B (return NULL when none remain):**
If no unresolved unknowns exist:
- Line 4022-4024: `else { newActiveUnknownNodeId = null; }`
- Line 4055/4063: deterministicSelection status is not "selected" → finalSelectedQuestion is null
- Line 4147/effectiveSelectedQuestion also becomes null
- Result.selectedQuestion at line 4377 returns null
- result.noQuestionReason = "No unresolved unknown candidates remain after this update."
This existing fallback chain works correctly for the `resolved` path via resolvedUnknownNodeIds. The gap is that `known` nodes bypass these checks because none of them verify terminal status against `["known", "resolved", "contradicted"]`.
---
## Checkpoint 4 — Canonical terminal-state predicate
**Best canonical rule:**
```
node.status not in ["resolved", "contradicted", "known"] && node.kind === "unknown"
```
**Why:**
- `status === "unknown"` alone is insufficient because it doesn't explicitly enumerate what counts as terminal, making the code fragile to future status additions.
- The explicit exclusion set `["resolved", "contradicted", "known"]` precisely captures all terminal states: resolved (explicitly resolved via reasoning), known (decision sufficiency reached), and contradicted (evidence contradicts). This is domain-general — it doesn't depend on which array the node happens to be in at a given moment.
- `resolvedNodeIds` alone is insufficient because `status === "known"` nodes are NOT added to resolvedNodeIds; they only get their status changed via updatedNodes.newStatus. Checking resolvedNodeIds alone would miss known-status nodes entirely.
---
## Checkpoint 5 — Scope of isSelectableUnresolvedUnknown
**Adding `known` exclusion to `isSelectableUnresolvedUnknown` alone:**
```
prevent 60B.37 stale final question:
PARTIAL — Would prevent the bug in Source B (line 3764 preservation), Source C (line 3984 model-selection honour), and Source F (selectPatternCompatibleUnknownCandidate at line 2186). Would NOT fix Source A (selectActiveUnknownCandidate has zero status check) or Source D (remainingUnknownExists has its own inline check with no reuse of isSelectableUnresolvedUnknown).
preserve valid pre-mutation proposal acceptance:
YES — isSelectableUnresolvedUnknown is only called post-mutation. validateSelectedQuestion at line 246 remains unchanged, so the customer-signing closure proposal would still pass validation before mutation.
preserve unresolved preferred target:
YES — genuine unknown-status nodes are not affected by adding "known" to the exclusion list. Only terminal nodes are excluded.
preserve prerequisite-first fallback:
YES — prerequisite blocking logic depends on hasUnresolvedSameProposalDependsOnPrerequisite (line 2209), which is independent of status filtering. Adding "known" to the exclusion preserves all existing unresolved targets.
Additional guard required:
YES — remainingUnknownExists at line 3705-3712 needs its own inline status check (or the entire source chain needs to converge through a single canonical predicate). Without it, Source A would still select a known node via selectActiveUnknownCandidate when no other unresolved candidates exist.
```
---
## Checkpoint 6 — Active unknown lifecycle
**Can known node remain active after only fixing final selectedQuestion:**
YES
Even if the final selectedQuestion is corrected to not target a known node, `newActiveUnknownNodeId` (line 3696-3703) would still be set from `validatedProposal.selectedQuestion.nodeId` at line 3702, and remainingUnknownExists at line 3705 would return TRUE for a known-status node because it only checks kind and resolvedNodeIds.
**Would that create observable lifecycle inconsistency:**
PARTIAL — The final selectedQuestion might be corrected (if we fix Source A), but newActiveUnknownNodeId on the graph object would still point to a known-status node, creating an inconsistent state where:
- updatedSituationGraph.activeUnknownNodeId points to a known node
- But result.selectedQuestion is null (or targets something else)
**Does the same post-mutation guard naturally correct both:**
YES — If we add `known` exclusion to `remainingUnknownExists` at line 3705-3712, then:
- For the known proposal-selected target: remainingUnknownExists returns false → newActiveUnknownNodeId gets reassigned via selectActiveUnknownCandidate (which would also need the fix). The fix propagates through the entire chain.
- Both activeUnknownNodeId and selectedQuestion would be corrected by the same boundary.
---
## Candidate Assessment
### Candidate A — `isSelectableUnresolvedUnknown` ONLY
Add "known" to the exclusion list at line 1663; leave validateSelectedQuestion unchanged.
```
60B.37 fixed: PARTIAL — Fixes Sources B and C but NOT Source A (selectActiveUnknownCandidate) or D (remainingUnknownExists). The known decision node would still be selected via Source A.
60B.11 preserved: YES
Prerequisite-first preserved: YES
Closure proposal remains valid: YES — pre-mutation validateSelectedQuestion unchanged
Active lifecycle coherent: NO — newActiveUnknownNodeId would still contain the known node via remainingUnknownExists gap.
Implementation surface: SMALL — one line change to exclusion list at line 1663 + same fix to line 2186 and line 1677 for consistency.
```
### Candidate B — POST-MUTATION PREFERRED-TARGET REVALIDATION
Immediately after applyGraphUpdate (line 3680), check proposal-selected target against updated graph before preserving it at lines 3701-3703 and 3764.
```
60B.37 fixed: YES — Guard at line 3704 would check status of proposal.target against ["known", "resolved", "contradicted"]. If known, remainingUnknownExists would correctly return false, triggering fallback to selectActiveUnknownCandidate. Combined with Source A fix, the final selectedQuestion and activeUnknownNodeId would both be corrected.
60B.11 preserved: YES — only terminal targets are excluded; genuine unknown-status preferred targets pass through unchanged.
Prerequisite-first preserved: YES — post-mutation revalidation checks status (terminality), not prerequisite deps. The hasUnresolvedSameProposalDependsOnPrerequisite check at line 3986 remains unaffected.
Closure proposal remains valid: YES — pre-mutation validation is untouched. The guard only runs on the already-committed updatedSituationGraph, after proposal acceptance.
Active lifecycle coherent: YES — same guard corrects both newActiveUnknownNodeId and deterministicSelection through the existing fallback chain.
Implementation surface: SMALL — guard at line 3704 checking status of validatedProposal.selectedQuestion.nodeId against updatedSituationGraph. Plus adding known to remainingUnknownExists inline check (line 3710).
```
### Candidate C — FINAL CONSTRUCTOR GUARD
Allow selection logic to proceed, but refuse to construct selectedQuestion for a terminal node at lines 4055/4147.
```
60B.37 fixed: PARTIAL — Could suppress the final question output, but deterministicSelection would still contain the known node ID. The graph object would have activeUnknownNodeId pointing to a known node. Observable inconsistency remains.
60B.11 preserved: YES
Prerequisite-first preserved: YES
Closure proposal remains valid: YES
Active lifecycle coherent: NO — deterministicSelection and activeUnknownNodeId both carry terminal target. Only the output question is suppressed, creating an inconsistent intermediate state.
Implementation surface: SMALL — one additional status check at lines 4055/4147 before constructing selectedQuestion.
```
### Candidate D — CANONICAL POST-MUTATION SELECTABILITY FOR BOTH ACTIVE + FINAL TARGET
Use one unresolved/selectable check after mutation for preferred target, active target, and final selectedQuestion without changing pre-mutation proposal validity.
This is effectively a synthesis of Candidates B and C with the rule applied to all three sources simultaneously:
```
60B.37 fixed: YES — All three sources (A, B, C) get corrected. The single canonical check is: node.status not in ["known", "resolved", "contradicted"]. Applied at line 3704 as a post-mutation guard on validatedProposal.selectedQuestion.nodeId against updatedSituationGraph. Then the existing fallback chain naturally handles the rest.
60B.11 preserved: YES
Prerequisite-first preserved: YES
Closure proposal remains valid: YES
Active lifecycle coherent: YES — both activeUnknownNodeId and selectedQuestion corrected through same boundary.
Implementation surface: MEDIUM — requires changes to isSelectableUnresolvedUnknown (line 1663), selectActiveUnknownCandidate (line 596 of utils.js), remainingUnknownExists inline check (line 3710-3711), carriedActiveUnknownStillUnresolved inline check (line 3850), and listUnresolvedUnknownCandidates/listEligibleUnknownCandidates (lines 1677, 1689).
```
### Candidate E — COMBINATION (MINIMUM)
Combine: (1) add "known" to isSelectableUnresolvedUnknown at line 1663, AND (2) add known exclusion to the remainingUnknownExists inline check at lines 3710-3711.
```
60B.37 fixed: PARTIAL — Fixes Sources B and D but NOT Source A. When remainingUnknownExists correctly returns false for a known target (Candidate E part 2), the fallback at line 3714-3720 calls selectActiveUnknownCandidate which still has zero status filtering (Source A). If no other candidates exist, this resolves to null (good), but if other candidates DO exist, they get selected (also good — but only because selectActiveUnknownCandidate happens to pick a different candidate that remains unknown). Edge case: if ALL remaining candidates are also terminal (rare but possible in cascading resolution), Source A would still select a known node.
60B.11 preserved: YES
Prerequisite-first preserved: YES
Closure proposal remains valid: YES
Active lifecycle coherent: PARTIAL — ActiveUnknownNodeId corrected by remainingUnknownExists fix, but deterministicSelection could still carry terminal node via Source A if all candidates happen to be unknown-kind with known status.
Implementation surface: SMALL-TWO-LINES — line 1663 and lines 3710-3711.
```
---
## Critical distinction
**Choice: B — PREFERRED TARGET NEEDS EXPLICIT POST-MUTATION REVALIDATION**
Why: The defect is not a general-purpose predicate gap (though that also exists). The core issue in 60B.37/38/40 is specifically that a **proposal-selected target** that transitions to `status=known` in the same turn survives as the final selectedQuestion because the post-mutation path re-purposes `validatedProposal.selectedQuestion.nodeId` as the default newActiveUnknownNodeId at line 3702 without verifying its status against the updated graph. The existing validation at line 3594 runs BEFORE mutation and sees the pre-mutation status. The fix must explicitly revalidate the proposal's selected target against post-mutation state, before it is preserved.
---
## Minimum corrective boundary
**Choice: E — MINIMUM COMBINATION**
Add known exclusion to two boundaries in sequence:
1. **isSelectableUnresolvedUnknown at line 1663** (add "known" to exclusion list) — fixes Sources B, C, F
2. **remainingUnknownExists inline check at lines 3710-3711** (add status !== "known" check) — fixes Source D
This combination:
- Fixes the exact 60B.37 defect (proposal target becomes known → remainingUnknownExists returns false → fallback to selectActiveUnknownCandidate or null)
- Preserves pre-mutation validation (no changes to validateSelectedQuestion)
- Preserves all valid unresolved preferred targets (only known/resolved/contradicted are excluded)
**Would another genuine unresolved unknown still be selectable:** YES — when remainingUnknownExists returns false, the fallback at line 3714 calls selectActiveUnknownCandidate which would pick the next highest-scored unknown-status node.
**Would no-question result occur when none remain:** YES — if selectActiveUnknownCandidate returns null (no unresolved candidates), newActiveUnknownNodeId becomes null and final selectedQuestion is null.
---
## Implementation readiness
**B — ONE MORE DESIGN QUESTION REQUIRED**
The minimum combination (Candidate E) would fix 60B.37 but leaves Source A (selectActiveUnknownCandidate at utils.js:596) with a residual gap. If all unknown-kind nodes in the graph happen to have status=known (cascading resolution edge case), selectActiveUnknownCandidate could incorrectly return null even though `eligibleCandidates` at line 3894 would also be empty (because listUnresolvedUnknownCandidates has the same gap). In practice this means:
1. **If there are other genuine unresolved unknowns:** The existing eligibleCandidates path (line 3894) + deterministicSelection fallback correctly handles it, but only by accident — if eligibleCandidates is built with the same gap, it might include known nodes too.
2. **The clean fix requires one additional boundary:** Either unify selectActiveUnknownCandidate through a canonical predicate OR add an inline status check alongside remainingUnknownExists at line 3710-3711.
**One unresolved question:**
Does the existing eligibleCandidates + deterministicSelection fallback chain (lines 3894-4024) already provide sufficient protection against selecting known-status nodes when other genuine candidates exist? If YES, then Candidate E (minimum combination) is sufficient. If NO — if selectActiveUnknownCandidate could return a known-status node as the "best" candidate even when eligibleCandidates is correctly filtered — then one additional boundary is needed.
**Smallest implementation boundary:**
Add status check to remainingUnknownExists at line 3710-3711 (fixes Source D / the direct survival of the proposal target as newActiveUnknownNodeId) + add "known" to isSelectableUnresolvedUnknown at line 1663 (fixes Sources B, C, F). Then verify whether selectActiveUnknownCandidate needs a parallel fix or whether the existing eligibleCandidates path already protects against it.
---
## Production code changed: NO
## Tests changed: NO
## Prompt changed: NO
## Schema changed: NO
## Ollama calls: 0
## Live API calls: 0
## Vitest run: NO
## Documentation updated: