7.5 KiB
Confidence Engine — Current Handoff
Repository position
- branch:
feature/decision-closure-ownership-v0.47 - checkpoint commit:
772ae49
Current green reasoning state
- substantive-tie active ownership: In complete unresolved ties among top-scoring candidates, the currently active node is preserved as the selection winner rather than falling through to stable label/display-order ordering. This only applies when the active node is eligible and remains substantively tied at the structural level.
- question-rejection active ownership: When a selected candidate's graph-backed question formulation is rejected as too complex (decomposition-required), the selected target node retains its ownership — it is not reseated to another candidate via
reseatSelectionAfterQuestionRejection. Instead, the selection remains on the original node withselectedQuestion = nulland an explicit rejection reason. - QUESTION_CONTINUATION: Existing question continuation logic remains intact and functional after the question-rejection fix.
- 60B.84: State B guard correctly does not fire for specific factor nodes merely because they are inside a decision context.
- State B: Sufficiency question path reaches expected terminal state without unintended firings.
- captured product-launch replay: The live product-launch start graph replays deterministically: ntpt9ki remains the active investigation target with no question selected after formulation rejection.
Latest resolved reasoning boundaries
1. Complete substantive selection tie
- previous behaviour: When all leading structural candidates were tied, the system always fell through to stable label/display-order as the final deterministic tie-breaker, regardless of which node was currently active in the investigation.
- corrected invariant: If the active node is among the tied structural candidates and remains eligible (unresolved, not contradicted), it is preserved as the winner. The stable label/display-order fallback is only used when there is no active candidate or when the active candidate does not remain among the top structural ties.
- regression location:
lib/graph/utils.js—classifyCandidateOrdering()now accepts anactiveNodeIdparameter and checks for active-tied candidates within the leading structural set before using display-order fallback. Callers inselectActiveUnknownCandidate()andexplainUnknownSelection()passgraph.activeUnknownNodeId. - regression test location:
tests/graph/utils.test.js— tests: "preserves the active candidate when it remains eligible and substantively tied", "transfers ownership when the active candidate substantively loses on score", "transfers ownership when the active candidate is resolved or ineligible"
2. Question-formulation rejection
- previous behaviour: When a selected candidate's question formulation was rejected (decomposition required),
determineGraphBackedQuestioncalledreseatSelectionAfterQuestionRejectionwithexcludedNodeIdsthat excluded the current target, causing investigation ownership to transfer to another candidate — even though the original target remained the strongest unresolved unknown. - corrected invariant: The selected node keeps its status and selection identity.
selectedQuestionis set tonullandquestionSuppressedReasonrecords the rejection explanation. No reseating occurs. - regression/captured fixture location:
tests/graph/orchestrator.test.js— "retains ownership when the strongest target's formulated question is rejected"; captured replay viatests/fixtures/live-product-launch-start-response.json
Current deterministic product-launch evidence
deterministicSelection.nodeId = ntpt9ki(active investigation target preserved)selectedQuestion = nullafter rejected formulation- explicit
noQuestionReason: "The selected investigation target remains active, but its current graph-backed question formulation was rejected as too complex." nxmeiabis not substituted in place of ntpt9ki
Current product meaning
activeUnknownNodeIdrepresents ongoing investigation ownership — it tracks which unknown candidate the system has committed to investigating.- Wording/formulation failure (question complexity / decomposition-required) does not itself invalidate the target. The target remains selected even when its formulated question cannot be answered in one step.
- Stable label ordering remains only a final fallback after substantive scoring, structural comparison, semantic signature checks, and tie/ownership handling are all exhausted.
Not yet proved
- live behavioural validation after these fixes (requires an actual product-launch run through the dev server)
- whether same-target reformulation is better than no-question (system currently uses no-question approach)
- broader/full-suite state beyond the six verified guards
- correctness of the question-complexity heuristic itself (that is a separate design concern)
Canonical live apparatus for next validation
scripts/reproduce-multi-turn-investigation.mjs- existing healthy dev server
.env.localauthoritative for:OLLAMA_BASE_URLOLLAMA_MODEL
- no model discovery
- no supplementary harnesses
- no direct Ollama calls
Next recommended step
- one observation-only live product-launch validation using the fixed scenario from the recorded journey
- no production changes during the experiment
- verify that the live LLM responds consistently to the null-question state and continues investigation on ntpt9ki
Experiment 60B.95 result (2026-08-17)
Classification: E — LIVE PATH DIVERGED
The live model selected nk6eyn2 ("exact monetary value of potential enterprise contract relative to £300k launch cost") as the investigation target, not npzfx36 ("likelihood, negotiation stage, and targeted signing date for the large enterprise customer"). Both are unresolved unknowns in the same scenario. An acceptable question was produced ("What outcome would demonstrate enough value to justify launching a software product now?"), so the question-rejection boundary was not reached.
What this establishes: The live engine can produce an acceptable graph-backed question on a fresh product-launch start without requiring decomposition.
What this does NOT prove: Whether investigation ownership is preserved when a selected target's formulation is rejected (the core invariant from checkpoint 60B.93). The question-rejection boundary was not reached because the live model chose a different investigation target with an acceptable question path.
Experiment 60B.97 result (2026-08-18)
Classification: E — START PATH DIVERGED
The live model again selected a non-financial-comparison target on the product-launch scenario. The Start selected enterprise-customer signing probability ("What evidence would clarify probability or likelihood that the enterprise customer will sign within the current launch window?") rather than the expected cash-flow / NPV comparison. Update 1 then hit a result_validation error: "Active unknown violates reasoning pattern consistency: "n_fincomp12" is explanation but active pattern is decision."
What this establishes: (1) The divergence from financial-comparison targets continues across multiple runs; (2) there exists an internal graph-state inconsistency (explanation-node vs decision-pattern mismatch in the Start/Update pipeline).
What this does NOT prove: Whether a financially-comparison-aligned Start would progress coherently with a financial answer, or whether the pattern consistency issue is deterministic.