experiment: define active selector terminal guard
This commit is contained in:
@@ -2964,3 +2964,7 @@ Experiment 60B.38 was a read-only diagnosis of why `n_product_launch_decision` b
|
||||
---
|
||||
|
||||
Experiment 60B.40 performed read-only post-mutation guard location diagnosis for same-turn terminal-target survival (known status). **Classification: B — PREFERRED TARGET NEEDS EXPLICIT POST-MUTATION REVALIDATION.** Traced all five post-mutation sources that can supply the final selectedQuestion node after applyGraphUpdate (line 3671): Source A (selectActiveUnknownCandidate, line 3722) has zero status filtering; Source B (isSelectableUnresolvedUnknown preservation at line 3764) excludes ["resolved", "contradicted"] but not "known"; Source C (model-selection honour at line 3984) has same gap; Source D (remainingUnknownExists inline check at line 3710) checks only kind + resolvedNodeIds, no status; Source E/F (selectPatternCompatibleUnknownCandidate and listUnresolvedUnknownCandidates) share the same ["resolved", "contradicted"] exclusion gap. Earliest safe guard point: line 3704 — updatedSituationGraph exists, proposal accepted, selected target can still be discarded, question not yet finalized. Fallback behaviour confirmed as C (both A+B): selectAnotherUnresolvedCandidate when one exists, return null when none remain. Minimum corrective boundary: E (minimum combination) — add "known" to isSelectableUnresolvedUnknown exclusion list AND add known exclusion to remainingUnknownExists inline check at line 3710-3711. Leaves residual gap in selectActiveUnknownCandidate if all candidates cascade to known status. One unresolved question requires clarification before readiness. Full analysis in docs/experiment-60b40.md. No production code changed. 0 Ollama calls. Pure code inspection.
|
||||
|
||||
---
|
||||
|
||||
Experiment 60B.41 was a read-only diagnosis of whether `selectActiveUnknownCandidate` must independently exclude terminal-status nodes (known/resolved/contradicted) for the 60B.37 closure path to be correct and fallback selection to remain semantically sound. **Classification: A — SELECTOR ITSELF MUST FILTER TERMINAL STATUS.** The selector at utils.js:593 filters only `kind === "unknown"` and `!resolvedNodeIds.includes(n.id)`. Zero status filtering exists. Key findings: (1) Scoring (`scoreUnknownCandidate`) is completely blind to node.status — it uses text classification, downstream count, and upstream unresolved dependency count, none of which inspect status. A known-status unknown-kind node scores identically to an unknown-status one. (2) In 60B.37's exact scenario: if no other genuine unresolved unknown remains, the selector would return the known decision node as the sole candidate, recreating the stale-target bug post-guard. (3) Adding terminal-status filtering does NOT affect genuine fallback candidates or prerequisite-first ordering because filtering happens before scoring and all genuine unknown-status nodes pass unchanged. (4) Five independent locations share the same gap pattern (`["resolved", "contradicted"]` exclusion without "known"): isSelectableUnresolvedUnknown, listUnresolvedUnknownCandidates, carriedActiveUnknownStillUnresolved, remainingUnknownExists inline check, and selectPatternCompatibleUnknownCandidate. This is PARTIAL overlap — different contracts serving different operational contexts but sharing the same systematic gap. **Critical distinction: D — SHARED ELIGIBILITY CONTRACT IS REQUIRED.** **Minimum implementation model: A — add terminal-status filter to selectActiveUnknownCandidate.** The selector must independently guard because it has zero caller-enforced input protection and known-status nodes bypass resolvedNodeIds entirely (they transition via updatedNodes.newStatus). Implementation boundary: one line change at utils.js:596 — add `!["known", "resolved", "contradicted"].includes(n.status)` to the existing filter. Ready for bounded implementation. No production code changed. 0 Ollama calls. Pure code inspection. Full trace in docs/experiment-60b41.md.
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
# Experiment 60B.41 — Does `selectActiveUnknownCandidate` need its own known-status guard?
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Branch:** `feature/known-target-exclusion-v0.36`
|
||||
**Objective:** Determine whether `selectActiveUnknownCandidate` must independently exclude `status = known` (and other terminal states) for the 60B.37 closure path to be correct and for fallback selection to remain semantically sound.
|
||||
|
||||
---
|
||||
|
||||
## Checkpoint 1 — Exact selector contract
|
||||
|
||||
```
|
||||
function: selectActiveUnknownCandidate(graph, resolvedNodeIds)
|
||||
location: lib/graph/utils.js:593-642
|
||||
candidate source: graph.nodes (all nodes in the graph)
|
||||
|
||||
kind filter: node.kind === "unknown"
|
||||
status filter: NONE — zero status filtering. The inline filter is:
|
||||
(n) => n.kind === "unknown" && !resolvedNodeIds.includes(n.id)
|
||||
resolvedNodeIds filter: !resolvedNodeIds.includes(n.id)
|
||||
|
||||
other eligibility filter: none — purely kind + resolvedNodeIds
|
||||
scoring happens after filtering: YES — scoreUnknownCandidate runs on the already-filtered unresolved set at line 603
|
||||
```
|
||||
|
||||
**Scoring function analysis** (`scoreUnknownCandidate`, utils.js:332-361):
|
||||
- `collectNodeText(node)` — node label/description text only
|
||||
- `classifyUnknownPriority(text)` — keyword classification on text
|
||||
- `findDependentNodes(graph, node.id).length` — downstream edge count
|
||||
- `countIncomingUnknownDependencies(graph, node.id, resolvedNodeIds)` — upstream dep count
|
||||
|
||||
None of these inspect `node.status`. A node's status field is completely invisible to scoring.
|
||||
|
||||
```
|
||||
Can status=known enter scoring: YES
|
||||
Can status=resolved enter scoring if absent from resolvedNodeIds: YES (theoretically possible via a bug in caller, but practically blocked by caller passing the correct resolvedNodeIds)
|
||||
Can status=contradicted enter scoring if absent from resolvedNodeIds: YES (same theoretical possibility as resolved)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Checkpoint 2 — 60B.37 fallback reconstruction
|
||||
|
||||
**Scenario:** `n_product_launch_decision` becomes terminal (`status=known`) during the same mutation turn. No other genuine unresolved unknown remains in the graph. The post-mutation guard correctly discards it as proposal target, then the fallback path runs:
|
||||
|
||||
```
|
||||
selectActiveUnknownCandidate(updatedSituationGraph, resolvedNodeIds)
|
||||
```
|
||||
|
||||
**Candidates seen:**
|
||||
- All `kind === "unknown"` nodes that are NOT in `resolvedNodeIds`
|
||||
- `n_product_launch_decision` has `kind === "unknown"` and is NOT in `resolvedNodeIds` (known-status nodes use `updatedNodes.newStatus`, not `resolvedUnknownNodeIds`)
|
||||
- Therefore `n_product_launch_decision` appears as the sole candidate
|
||||
|
||||
**Would `n_product_launch_decision` still qualify:** YES — passes both filters: kind="unknown" ✓, not in resolvedNodeIds ✓
|
||||
|
||||
**Would it be returned:** YES — with no other candidates to compete against, it scores highest by default (only candidate). Without terminal-status filtering, its status is invisible to scoring and classification.
|
||||
|
||||
**Would final selectedQuestion become non-null again:** YES — `newActiveUnknownNodeId` would be set to the known node's ID at line 3716-3719, and this would propagate through deterministicSelection → finalSelectedQuestion → result.selectedQuestion, recreating the 60B.37 stale-target bug exactly.
|
||||
|
||||
---
|
||||
|
||||
## Checkpoint 3 — Genuine fallback case
|
||||
|
||||
**Existing test/case:** `reproduce-multi-turn-investigation.harness.test.js:1388` (pre-anchored product-launch customer-signing fixture)
|
||||
- Nodes: `n_product_launch_decision` (kind=unknown, status=unknown), `n_enterprise_customer_signing` (kind=unknown, status=unknown)
|
||||
- This is a genuine two-candidate scenario
|
||||
|
||||
**Remaining unresolved candidate:** `n_enterprise_customer_signing` (status=unknown, kind=unknown, not resolved)
|
||||
|
||||
**Would known-status exclusion affect it:** NO — this node has `status === "unknown"`, so adding terminal-status filtering to the selector would still let it pass all filters. Its scoring is identical because status doesn't enter scoring logic.
|
||||
|
||||
**Would prerequisite-first ordering change:** NO — prerequisite blocking depends on `hasUnresolvedSameProposalDependsOnPrerequisite` (apply-proposal.js:2209) which checks node.kind membership in `proposal.dependsOn`. This is independent of node status. No known-status exclusion could alter prerequisite-first ordering because it operates at the kind+resolved boundary, not the prerequisite boundary.
|
||||
|
||||
---
|
||||
|
||||
## Checkpoint 4 — Duplicated eligibility logic
|
||||
|
||||
**Choice:** PARTIAL — OVERLAPPING BUT DIFFERENT CONTRACTS
|
||||
|
||||
**Why:** `isSelectableUnresolvedUnknown` and `selectActiveUnknownCandidate` share the same *intent* (find unresolved unknown nodes) but differ in their terminal-state handling: the predicate excludes `["resolved", "contradicted"]` while the selector has zero status filtering. However, they also serve different operational contexts — the predicate validates a single node ID by reference (used for preservation checks), while the selector enumerates and ranks all candidates from the graph. `listUnresolvedUnknownCandidates` shares the predicate's exclusion list. `carriedActiveUnknownStillUnresolved` mirrors the predicate's pattern inline. None of these functions treat "known" as terminal, creating a systematic gap across all five locations rather than a pure duplication.
|
||||
|
||||
---
|
||||
|
||||
## Checkpoint 5 — Canonical rule placement
|
||||
|
||||
### Candidate A — PATCH SELECTOR ONLY
|
||||
|
||||
Add terminal-status exclusion directly inside `selectActiveUnknownCandidate`.
|
||||
|
||||
```
|
||||
60B.37 safe: YES — The fallback candidate would exclude known/resolved/contradicted, preventing stale target re-selection.
|
||||
Can return terminal nodes elsewhere: YES — isSelectableUnresolvedUnknown (line 1663), remainingUnknownExists inline (line 3710-3711), listUnresolvedUnknownCandidates (line 1677), carriedActiveUnknownStillUnresolved (line 3850) all have the same gap.
|
||||
Preserves existing scoring: YES — status filtering is applied before scoring; nodes that already pass kind+resolved filters retain their scores unchanged. Adding one more filter cannot change relative ordering.
|
||||
Semantic-drift risk: MEDIUM — fixes only one of five locations; other gaps remain silently active.
|
||||
Implementation scope: SMALL — one line change inside the existing filter at utils.js:596.
|
||||
```
|
||||
|
||||
### Candidate B — REUSE CANONICAL PREDICATE
|
||||
|
||||
Make selector candidate eligibility equivalent to `isSelectableUnresolvedUnknown` or a shared helper with the same terminal-state semantics.
|
||||
|
||||
```
|
||||
60B.37 safe: YES — Same correctness as Candidate A, but also fixes Sources D, E, F, G identified in 60B.40.
|
||||
Can return terminal nodes elsewhere: NO — all five locations converge on the same canonical rule.
|
||||
Preserves existing scoring: YES — filtering scope expands uniformly; no node's relative score changes.
|
||||
Semantic-drift risk: LOW — eliminates the systematic gap across all paths, establishing a single source of truth for unresolved unknown eligibility.
|
||||
Implementation scope: MEDIUM — requires changes to utils.js (selector) AND apply-proposal.js (remainingUnknownExists, carriedActiveUnknownStillUnresolved, listUnresolvedUnknownCandidates), plus updating isSelectableUnresolvedUnknown to include "known".
|
||||
```
|
||||
|
||||
### Candidate C — LEAVE SELECTOR UNCHANGED
|
||||
|
||||
Rely on callers/eligible-candidate chains to protect it.
|
||||
|
||||
```
|
||||
60B.37 safe: PARTIAL — Would work only if remainingUnknownExists at line 3710-3711 is also fixed AND no other code path reaches the selector with a known candidate in its filter set. But the 60B.40 analysis (Sources D, F, G) shows multiple inline checks also have the gap.
|
||||
Can return terminal nodes elsewhere: YES — Sources B (isSelectableUnresolvedUnknown), F (selectPatternCompatibleUnknownCandidate), and G (listUnresolvedUnknownCandidates) all pass known-status through.
|
||||
Preserves existing scoring: LIKELY — unchanged selector preserves current behavior; risk is in unguarded callers, not the selector itself.
|
||||
Semantic-drift risk: HIGH — relies on fragile assumption that callers always provide correct filtered input. No defense-in-depth.
|
||||
Implementation scope: SMALL (selector side) / LARGE (to actually fix — would require fixing all callers).
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Critical distinction
|
||||
|
||||
**Choice: D — SHARED ELIGIBILITY CONTRACT IS REQUIRED**
|
||||
|
||||
Why: The gap (`known` not treated as terminal) exists across five independent locations with identical filtering logic. Fixing only one is a band-aid; the remaining four continue to silently accept known-status nodes as eligible unresolved unknowns. A shared predicate eliminates the systematic inconsistency at its root rather than treating each symptom individually.
|
||||
|
||||
---
|
||||
|
||||
## Minimum implementation model
|
||||
|
||||
**Choice: A — add terminal-status filter to selectActiveUnknownCandidate**
|
||||
|
||||
Why: For the specific question of this experiment (does 60B.41 require a fix to the selector itself?), the answer is definitively YES. The selector MUST independently exclude terminal statuses because:
|
||||
1. It has zero status filtering today — the only filters are kind and resolvedNodeIds
|
||||
2. Known-status nodes bypass resolvedNodeIds (they use updatedNodes.newStatus, not resolvedUnknownNodeIds)
|
||||
3. No caller guarantees filtered input before reaching the selector
|
||||
4. Adding `status !== "known" && status !== "resolved" && status !== "contradicted"` to the filter prevents 60B.37 without affecting any genuine unresolved candidate
|
||||
|
||||
**Would genuine unresolved fallback still work:** YES — genuine unknown-status nodes pass all filters unchanged. Their scoring is identical (status doesn't enter scoring). Prerequisite-first ordering is unaffected.
|
||||
|
||||
**Would prerequisite-first scoring remain unchanged:** YES — filtering adds a gate before scoring, not during it. No node's score or rank changes; only the candidate set shrinks by removing terminal nodes that would have been invisible to scoring anyway.
|
||||
|
||||
**Would valid closure proposals remain accepted:** YES — `validateSelectedQuestion` (pre-mutation validation) is untouched. The filter only applies post-mutation selection. A customer-signing closure proposal that was valid pre-mutation still passes all filters post-mutation because the node's status hasn't changed.
|
||||
|
||||
---
|
||||
|
||||
## Implementation readiness
|
||||
|
||||
**Choice: A — READY FOR BOUNDED IMPLEMENTATION**
|
||||
|
||||
The question is answered definitively. The selector must add terminal-status filtering. One unresolved follow-on question remains for separate treatment: whether `isSelectableUnresolvedUnknown` and other predicate functions also need `"known"` added to their exclusion lists (they do, but that is a scope decision beyond 60B.41).
|
||||
|
||||
**Smallest implementation boundary:**
|
||||
Add status filter to `selectActiveUnknownCandidate` at utils.js:596. Change line 596 from:
|
||||
```js
|
||||
(n) => n.kind === "unknown" && !resolvedNodeIds.includes(n.id)
|
||||
```
|
||||
to:
|
||||
```js
|
||||
(n) => n.kind === "unknown" &&
|
||||
!["known", "resolved", "contradicted"].includes(n.status) &&
|
||||
!resolvedNodeIds.includes(n.id)
|
||||
```
|
||||
|
||||
Production code changed: NO
|
||||
Tests changed: NO
|
||||
Prompt changed: NO
|
||||
Schema changed: NO
|
||||
Ollama calls: 0
|
||||
Live API calls: 0
|
||||
Vitest run: NO
|
||||
Reference in New Issue
Block a user