71 lines
6.5 KiB
Markdown
71 lines
6.5 KiB
Markdown
# Confidence Engine — Current Handoff
|
|
|
|
## Repository position
|
|
- branch: `feature/decision-closure-ownership-v0.47`
|
|
- checkpoint commit: `772ae49`
|
|
|
|
## Current green reasoning state
|
|
- **substantive-tie active ownership**: In complete unresolved ties among top-scoring candidates, the currently active node is preserved as the selection winner rather than falling through to stable label/display-order ordering. This only applies when the active node is eligible and remains substantively tied at the structural level.
|
|
- **question-rejection active ownership**: When a selected candidate's graph-backed question formulation is rejected as too complex (decomposition-required), the selected target node retains its ownership — it is not reseated to another candidate via `reseatSelectionAfterQuestionRejection`. Instead, the selection remains on the original node with `selectedQuestion = null` and an explicit rejection reason.
|
|
- **QUESTION_CONTINUATION**: Existing question continuation logic remains intact and functional after the question-rejection fix.
|
|
- **60B.84**: State B guard correctly does not fire for specific factor nodes merely because they are inside a decision context.
|
|
- **State B**: Sufficiency question path reaches expected terminal state without unintended firings.
|
|
- **captured product-launch replay**: The live product-launch start graph replays deterministically: ntpt9ki remains the active investigation target with no question selected after formulation rejection.
|
|
|
|
## Latest resolved reasoning boundaries
|
|
|
|
### 1. Complete substantive selection tie
|
|
- previous behaviour: When all leading structural candidates were tied, the system always fell through to stable label/display-order as the final deterministic tie-breaker, regardless of which node was currently active in the investigation.
|
|
- corrected invariant: If the active node is among the tied structural candidates and remains eligible (unresolved, not contradicted), it is preserved as the winner. The stable label/display-order fallback is only used when there is no active candidate or when the active candidate does not remain among the top structural ties.
|
|
- regression location: `lib/graph/utils.js` — `classifyCandidateOrdering()` now accepts an `activeNodeId` parameter and checks for active-tied candidates within the leading structural set before using display-order fallback. Callers in `selectActiveUnknownCandidate()` and `explainUnknownSelection()` pass `graph.activeUnknownNodeId`.
|
|
- regression test location: `tests/graph/utils.test.js` — tests: "preserves the active candidate when it remains eligible and substantively tied", "transfers ownership when the active candidate substantively loses on score", "transfers ownership when the active candidate is resolved or ineligible"
|
|
|
|
### 2. Question-formulation rejection
|
|
- previous behaviour: When a selected candidate's question formulation was rejected (decomposition required), `determineGraphBackedQuestion` called `reseatSelectionAfterQuestionRejection` with `excludedNodeIds` that excluded the current target, causing investigation ownership to transfer to another candidate — even though the original target remained the strongest unresolved unknown.
|
|
- corrected invariant: The selected node keeps its status and selection identity. `selectedQuestion` is set to `null` and `questionSuppressedReason` records the rejection explanation. No reseating occurs.
|
|
- regression/captured fixture location: `tests/graph/orchestrator.test.js` — "retains ownership when the strongest target's formulated question is rejected"; captured replay via `tests/fixtures/live-product-launch-start-response.json`
|
|
|
|
## Current deterministic product-launch evidence
|
|
- `deterministicSelection.nodeId = ntpt9ki` (active investigation target preserved)
|
|
- `selectedQuestion = null` after rejected formulation
|
|
- explicit `noQuestionReason`: "The selected investigation target remains active, but its current graph-backed question formulation was rejected as too complex."
|
|
- `nxmeiab` is not substituted in place of ntpt9ki
|
|
|
|
## Current product meaning
|
|
- `activeUnknownNodeId` represents ongoing investigation ownership — it tracks which unknown candidate the system has committed to investigating.
|
|
- Wording/formulation failure (question complexity / decomposition-required) does not itself invalidate the target. The target remains selected even when its formulated question cannot be answered in one step.
|
|
- Stable label ordering remains only a final fallback after substantive scoring, structural comparison, semantic signature checks, and tie/ownership handling are all exhausted.
|
|
|
|
## Not yet proved
|
|
- live behavioural validation after these fixes (requires an actual product-launch run through the dev server)
|
|
- whether same-target reformulation is better than no-question (system currently uses no-question approach)
|
|
- broader/full-suite state beyond the six verified guards
|
|
- correctness of the question-complexity heuristic itself (that is a separate design concern)
|
|
|
|
## Canonical live apparatus for next validation
|
|
- `scripts/reproduce-multi-turn-investigation.mjs`
|
|
- existing healthy dev server
|
|
- `.env.local` authoritative for:
|
|
- `OLLAMA_BASE_URL`
|
|
- `OLLAMA_MODEL`
|
|
- no model discovery
|
|
- no supplementary harnesses
|
|
- no direct Ollama calls
|
|
|
|
## Next recommended step
|
|
- one observation-only live product-launch validation using the fixed scenario from the recorded journey
|
|
- no production changes during the experiment
|
|
- verify that the live LLM responds consistently to the null-question state and continues investigation on ntpt9ki
|
|
|
|
---
|
|
|
|
## Experiment 60B.95 result (2026-08-17)
|
|
|
|
**Classification: E — LIVE PATH DIVERGED**
|
|
|
|
The live model selected nk6eyn2 ("exact monetary value of potential enterprise contract relative to £300k launch cost") as the investigation target, not npzfx36 ("likelihood, negotiation stage, and targeted signing date for the large enterprise customer"). Both are unresolved unknowns in the same scenario. An acceptable question was produced ("What outcome would demonstrate enough value to justify launching a software product now?"), so the question-rejection boundary was not reached.
|
|
|
|
**What this establishes:** The live engine can produce an acceptable graph-backed question on a fresh product-launch start without requiring decomposition.
|
|
|
|
**What this does NOT prove:** Whether investigation ownership is preserved when a selected target's formulation is rejected (the core invariant from checkpoint 60B.93). The question-rejection boundary was not reached because the live model chose a different investigation target with an acceptable question path.
|