# Confidence Engine — Current Handoff ## Repository position - branch: `feature/decision-closure-ownership-v0.47` - checkpoint commit: `772ae49` ## Current green reasoning state - **substantive-tie active ownership**: In complete unresolved ties among top-scoring candidates, the currently active node is preserved as the selection winner rather than falling through to stable label/display-order ordering. This only applies when the active node is eligible and remains substantively tied at the structural level. - **question-rejection active ownership**: When a selected candidate's graph-backed question formulation is rejected as too complex (decomposition-required), the selected target node retains its ownership — it is not reseated to another candidate via `reseatSelectionAfterQuestionRejection`. Instead, the selection remains on the original node with `selectedQuestion = null` and an explicit rejection reason. - **QUESTION_CONTINUATION**: Existing question continuation logic remains intact and functional after the question-rejection fix. - **60B.84**: State B guard correctly does not fire for specific factor nodes merely because they are inside a decision context. - **State B**: Sufficiency question path reaches expected terminal state without unintended firings. - **captured product-launch replay**: The live product-launch start graph replays deterministically: ntpt9ki remains the active investigation target with no question selected after formulation rejection. ## Latest resolved reasoning boundaries ### 1. Complete substantive selection tie - previous behaviour: When all leading structural candidates were tied, the system always fell through to stable label/display-order as the final deterministic tie-breaker, regardless of which node was currently active in the investigation. - corrected invariant: If the active node is among the tied structural candidates and remains eligible (unresolved, not contradicted), it is preserved as the winner. The stable label/display-order fallback is only used when there is no active candidate or when the active candidate does not remain among the top structural ties. - regression location: `lib/graph/utils.js` — `classifyCandidateOrdering()` now accepts an `activeNodeId` parameter and checks for active-tied candidates within the leading structural set before using display-order fallback. Callers in `selectActiveUnknownCandidate()` and `explainUnknownSelection()` pass `graph.activeUnknownNodeId`. - regression test location: `tests/graph/utils.test.js` — tests: "preserves the active candidate when it remains eligible and substantively tied", "transfers ownership when the active candidate substantively loses on score", "transfers ownership when the active candidate is resolved or ineligible" ### 2. Question-formulation rejection - previous behaviour: When a selected candidate's question formulation was rejected (decomposition required), `determineGraphBackedQuestion` called `reseatSelectionAfterQuestionRejection` with `excludedNodeIds` that excluded the current target, causing investigation ownership to transfer to another candidate — even though the original target remained the strongest unresolved unknown. - corrected invariant: The selected node keeps its status and selection identity. `selectedQuestion` is set to `null` and `questionSuppressedReason` records the rejection explanation. No reseating occurs. - regression/captured fixture location: `tests/graph/orchestrator.test.js` — "retains ownership when the strongest target's formulated question is rejected"; captured replay via `tests/fixtures/live-product-launch-start-response.json` ## Current deterministic product-launch evidence - `deterministicSelection.nodeId = ntpt9ki` (active investigation target preserved) - `selectedQuestion = null` after rejected formulation - explicit `noQuestionReason`: "The selected investigation target remains active, but its current graph-backed question formulation was rejected as too complex." - `nxmeiab` is not substituted in place of ntpt9ki ## Current product meaning - `activeUnknownNodeId` represents ongoing investigation ownership — it tracks which unknown candidate the system has committed to investigating. - Wording/formulation failure (question complexity / decomposition-required) does not itself invalidate the target. The target remains selected even when its formulated question cannot be answered in one step. - Stable label ordering remains only a final fallback after substantive scoring, structural comparison, semantic signature checks, and tie/ownership handling are all exhausted. ## Not yet proved - live behavioural validation after these fixes (requires an actual product-launch run through the dev server) - whether same-target reformulation is better than no-question (system currently uses no-question approach) - broader/full-suite state beyond the six verified guards - correctness of the question-complexity heuristic itself (that is a separate design concern) ## Canonical live apparatus for next validation - `scripts/reproduce-multi-turn-investigation.mjs` - existing healthy dev server - `.env.local` authoritative for: - `OLLAMA_BASE_URL` - `OLLAMA_MODEL` - no model discovery - no supplementary harnesses - no direct Ollama calls ## Next recommended step - one observation-only live product-launch validation using the fixed scenario from the recorded journey - no production changes during the experiment - verify that the live LLM responds consistently to the null-question state and continues investigation on ntpt9ki ## Apparatus correction: 60B.101 — null-question start capture The canonical `startOnly` harness was corrected to accept successful Start responses with `selectedQuestion = null`. Previously, any successful Start returning no graph-backed question (legitimate outcome meaning "target remains active but no askable question available") caused the harness to block and fail. **Change:** The harness now checks `success === true` + valid `situationGraph` as the sole gate for startOnly success. `selectedQuestion` is preserved exactly (including null) in the continuation state file without coercion. **Impact on 60B.100:** The evidence from 60B.100 was captured via direct curl because the harness blocked on null-question Start. That evidence is now marked as apparatus-contaminated and provisional observation only. --- ## Canonical harness gated apparatus (60B.99) The canonical harness (`scripts/reproduce-multi-turn-investigation.mjs`) now supports a two-phase gated investigation pattern: **startOnly** — `FIXTURE_MODE=startOnly` - Makes exactly one `/api/cases/start` request - Writes the captured Start state (graph + selectedQuestion) to `.evidence-temp/continuation-start-only.json` (or path set by `CONTINUATION_FILE`) - Issues zero Update requests - Exits successfully **continueOneUpdate** — `FIXTURE_MODE=continueOneUpdate CONTINUATION_ANSWER=` - Loads the persisted Start continuation state - Requires explicit answer (blocks with exit code 1 if missing) - Makes exactly one `/api/cases/update` using preserved Start state + explicit answer - Issues zero Start requests - Exits **Normal mode** (`FIXTURE_MODE` unset) — unchanged. Start → configured Update loop still works identically to pre-60B.99. This apparatus corrects the apparatus defect proven in 60B.98: the canonical harness can now stop after Start, allow external semantic inspection of the returned question, and later continue from that exact captured state with an explicitly chosen answer. --- ## Experiment 60B.95 result (2026-08-17) **Classification: E — LIVE PATH DIVERGED** The live model selected nk6eyn2 ("exact monetary value of potential enterprise contract relative to £300k launch cost") as the investigation target, not npzfx36 ("likelihood, negotiation stage, and targeted signing date for the large enterprise customer"). Both are unresolved unknowns in the same scenario. An acceptable question was produced ("What outcome would demonstrate enough value to justify launching a software product now?"), so the question-rejection boundary was not reached. **What this establishes:** The live engine can produce an acceptable graph-backed question on a fresh product-launch start without requiring decomposition. **What this does NOT prove:** Whether investigation ownership is preserved when a selected target's formulation is rejected (the core invariant from checkpoint 60B.93). The question-rejection boundary was not reached because the live model chose a different investigation target with an acceptable question path. ## Experiment 60B.97 result (2026-08-18) **Classification: E — START PATH DIVERGED** The live model again selected a non-financial-comparison target on the product-launch scenario. The Start selected enterprise-customer signing probability ("What evidence would clarify probability or likelihood that the enterprise customer will sign within the current launch window?") rather than the expected cash-flow / NPV comparison. **Valid evidence retained:** Start = S2 — DIFFERENT START (the live model diverged from the expected financial-comparison path). **Update 1 evidence: DISCARDED.** The canonical harness auto-continued with its preconfigured `answers[0]`, so the Update occurred outside the experiment's semantic gate. This was an apparatus defect (60B.98) — the harness did not provide a post-Start stop gate at that time. The HTTP 500 is NOT established as a reasoning defect from 60B.97. **Apparatus correction:** See section "Canonical harness gated apparatus (60B.99)" above for the fix. ## Experiment 60B.100 result (2026-08-18) **Classification: B — DETERMINISTIC SELECTOR OVERRIDES MODEL QUESTION** On a fresh product-launch Start, the LLM reconstruction question targeted one uncertainty ("What is the estimated probability that the large enterprise customer will sign?") while the deterministic graph-backed selector chose another ("What evidence would clarify the exact percentage of total projected revenue attributable to the enterprise customer?"). These are materially different: one asks about deal timing/commitment probability, the other asks about financial proportion/magnitude. The override was produced by fixed `actor_match` keyword scoring: node n65sgyd's label contained "enterprise customer" which matched the actor dictionary (+10 delta), giving it a decisive score of 10 vs 4 for both competitors. No tie/fallback was involved — the winner was determined entirely by keyword rule weighting. **What this establishes:** On fresh Start calls, deterministic keyword signals can override model-inferred investigation priority when node labels differ in dictionary-match patterns. The final investigation target is not the model's contextual judgment but the highest-scoring candidate under fixed scoring rules. **What this does NOT prove:** Whether the deterministic selection is better or worse than the model's suggestion; consistency across scenario types; or downstream investigation quality impact. --- ## RETURN-TO-ORIGIN CHECKPOINT **selector-led compulsory next-question optimisation is PAUSED** **semantic-selector replacement is also PAUSED** Recent work is preserved as valuable experimental learning. The graph/reconstruction/decomposition/invariant work remains potentially reusable. No replacement architecture has been selected. The next phase starts from the workspace/methodology question, not from a preselected technical solution. ### Required methodology-continuity context (read first) > **Before continuing any Confidence Engine reasoning development, load these in order:** > > 1. `docs/current-working-principles.md` §0 (Axiomatic Principles A1–A12) > 2. `docs/Confidence_Engine_Return_to_Origin_Methodology_Context_2026-08-18.md` **The axiomatic principles in current-working-principles.md are now the durable methodological anchor.** They codify delivery-platform independence, meaning-over-dictionary reasoning, user-directed non-linear investigation, provisional closure, and progressive disclosure. All future work must be evaluated against these 12 axioms. The RTO methodology context document covers: - What Confidence Engine is (reasoning method, not software or LLM) - What it is not - The granular answer-fragment hypothesis (working — not architecture) - RTO.14–17 learning and why they did not define the next architecture - Memory/graph separation principle - Optional capability: "Help me answer" / "Answer for me" - Methodology test for future development - Recent experimental evidence (RTO.18, RTO.20–RTO.22) supporting durable principles **Do not begin reasoning development from the latest experiment apparatus.** Always load this methodology continuity context first. Without it, a new session will reconstruct the project from recent technical experiments and lose sight of the method the application is meant to embody. ### Current Git checkpoint - **Branch:** `feature/decision-closure-ownership-v0.47` - **HEAD:** `e1b54e407389c7f1c25ba20645540d404b9e9df9` - **Latest commit:** `e1b54e4 test(experiment): checkpoint granular answer fragment apparatus` ### Status of RTO.14–17 RTO.14–17 experiments proved that an LLM can retain focused knowledge and carry coherent reasoning across turns inside a single investigation. This was valuable evidence. **However, they also revealed that the cumulative reconstruction pattern was reproduced at a lower scope.** Do not treat RTO.14–17 as the automatic architecture direction to continue. The next reasoning direction is the granular answer-fragment hypothesis: independent user-chosen question/answer deconstruction without accumulated focused state being passed between LLM calls. **Prohibition:** Do not automatically continue into cumulative-state compression or fourth-turn experiments without first revisiting the methodology continuity document above. See also: - `docs/methodology-checkpoint-return-to-origin.md` — repository-facing checkpoint summary --- ## Focused Semantic Deconstruction — Reasoning Status (2026-08-24) ### Current state - **Branch:** `feature/decision-closure-ownership-v0.47` - **HEAD:** `2c108df5a92ec4cb0c4fa7878002cb7eb0aea4fd` - **Working tree:** ``` M lib/graph/focused-investigation.js ?? tests/graph/live-focused-deconstruct-experiment-helper.mjs ``` - No unexpected temporary files or evidence directories. ### Canonical focused helper ``` tests/graph/live-focused-deconstruct-experiment-helper.mjs ``` Exports `runLiveFocusedDeconstructExperiment`. Future focused semantic experiments should: import it directly; use the real production `buildFocusedDeconstructPrompt`; use the production validator/provider through the helper; configure `.env.local` for `OLLAMA_BASE_URL` and `OLLAMA_MODEL`; make the explicitly budgeted number of live calls; never inline prompt/provider/schema logic; never replace the helper ad hoc. ### Product flow context ``` initial scenario → initial/global SituationGraph → user selects original open question/thread → focused formulate → user answer → focused deconstruct → contribution retained against original targetNodeId → subsequent focused formulate/deconstruct rounds ``` Focused deconstruction accumulates focused contributions rather than rewriting the initial global SituationGraph. Reuse/integration of accumulated focused learning is a later question, not part of this checkpoint. ### Semantic rules currently under review (validated in the cumulative diff) The production `buildFocusedDeconstructPrompt` encodes six semantic principles: 1. **Observation fidelity** — only meaning directly supported by what the user's answer states becomes an observation; do not strengthen implications into observations. 2. **Uncertainty fidelity** — preserve uncertainty at the narrowest scope justified by evidence; when one factor is established but nothing else is, keep remaining uncertainty broad rather than inventing specific additional factors/deficits/causes/interventions. 3. **Assumption attribution** — attribute only propositions that the user's answer would cease to make sense if false; do not import plausible interpretations from investigation context or model analysis into the assumptions field; return `assumptions: []` when no genuine assumption exists. 4. **Relationship strength** — only connections the user's answer directly establishes become relationships; co-mentioned facts do not create causal/constraint/dependency links. 5. **Proposition ownership** — track who owns each proposition (user's comfort, willingness, threshold, belief, judgement, etc.); do not elevate subjective stance into objective requirement or situational constraint. 6. **Evidence-first / action-appropriate follow-ups** — before formulating follow-ups, check whether the question tests a proposition against current epistemic state; if an explanation/deficit/cause/intervention has not been established by evidence, phrase the question to test whether that proposition is true rather than assuming it; do not jump to implementation details until intervention is the relevant next issue. A supporting negative principle also holds: *Do not solve semantic interpretation with an ever-growing deterministic dictionary/regex system. Structure may be deterministic; meaning remains semantic/LLM-based.* ### Validated learning (bounded evidence) Bounded focused experiments on this branch showed: - The original real UI failure was reproduced on the actual focused-deconstruct path. - Observations could remain faithful while assumptions/relationships were weakened beyond what the user stated. - Tightening the assumptions contract materially reduced model-generated assumptions being attributed to the user, supported by bounded contrast. - Genuine implicit dependencies can still be preserved through the tightened boundary. - Proposition ownership preserves subjective criteria without weakening genuine factual reports. - Tentative factual reports can retain their uncertainty. - Evidence-first follow-ups avoid premature remedy; practical follow-ups remain possible once a blocker is established. - Uncertainty specificity follows user-supplied evidence rather than defaulting to generic broadness. **Do not treat these as generalisation:** each finding is supported by the bounded contrast of tested cases only. ### EXP13 status: CONTAMINATED ``` EXP13 - CONTAMINATED — excluded from evidence ``` Reasons: iterative production prompt refinement; best-supported live-call count = 4; not valid bounded experiment evidence; tuned EXP13 uncertainty wording was removed during the diff. EXP13 must not be cited as proof of uncertainty behaviour. ### EXP13R / EXP14 (valid but conservative) - **EXP13R:** materially reduced invented uncertainty specificity but retained some residual operational specialisation. - **EXP14:** showed the same rule could preserve explicit user-supplied BACS/failed-payment specificity while leaving unidentified additional gaps broad. Do not conclude that the uncertainty boundary is fully solved. The contrast between these two cases does not establish generalisation. ### Current restart point The immediate semantic pass has established a coherent working principle: > **The Engine should preserve what was said, how certain it was, who owns the proposition, and how strongly evidence supports relationships; then investigate missing evidence before inventing explanations or remedies.** Deterministic tests (22/22) verify the structural boundaries. Live-call evidence is bounded. No new experiment is run in this checkpoint. --- ## LIVE MANUAL WALKTHROUGH FINDING — Focused Progression UI (2026-08-26) ### Verified during repaired Run A multi-turn focused-investigation flow The following was established by live manual walkthrough: ``` Run A presentation entry routing: FIXED Run A focused lifecycle continuity: FIXED focused question/form/response rendering inside Run A: FIXED same-node completed result reopen: FIXED multi-turn focused continuation: WORKING ``` ### KNOWN UI PROGRESSION / OWNERSHIP DEFECT — Follow-up Selected-Question Duplication During the repaired Run A multi-turn focused-investigation flow: ```text focused answer → deconstruction → "Questions this raises" → user clicks the single proposed follow-up → that follow-up becomes the active QUESTION above the textarea ``` The UI then continues to show the same previous proposal below under: ```text QUESTIONS THIS RAISES [same question] → pick this question ``` So the selected follow-up is simultaneously displayed as: ```text ACTIVE QUESTION ``` and: ```text STILL-AVAILABLE ACTIONABLE FOLLOW-UP ``` This is visibly duplicated and demonstrates that the **proposal state** and **active-question state** are disconnected. **Product interpretation:** This is a KNOWN UI PROGRESSION / OWNERSHIP DEFECT, not merely a styling issue. The useful semantic progression is: ```text answer → what this tells us → still unclear → proposed next question → user selects it → selected question becomes active ``` Once selected, its old representation must not continue to invite: ```text → pick this question ``` The old proposal should either become non-actionable provenance/history or cease being displayed as a candidate. Do not decide the final UI treatment in this checkpoint. ### Related observation — Single follow-up / plural-candidate ownership Record also that `possibleFollowUpQuestions` currently contains **exactly one** proposed question, while the user-facing heading says **"Questions this raises"** and presents it as a selectable candidate. This raises a later product/ownership question: > Is this genuinely a menu of candidate questions, or is it the engine's proposed continuation of the current investigation? Do not resolve that question now. ### Status of remaining items after this walkthrough ``` follow-up selected-question duplication: OPEN UI ISSUE focused → global ownership/integration: STILL OPEN Current Understanding incorporating focused learning: STILL OPEN ``` ### Important separation note The latest second-turn semantic output exposed a separate reasoning issue around assumption attribution/frontier prioritisation. This must not be conflated with the UI duplication defect described above. They are orthogonal concerns — one is presentation state management, the other is semantic reasoning fidelity. --- ## CONTRIBUTIONS → FINDINGS DOMAIN MODEL (2026-08-26) ### Discovery scope This section records a domain model discovery based on live manual walkthrough evidence across multiple focused deconstruction turns. It determines whether the Confidence Engine needs an explicit distinction between **Contribution**, **Finding**, and **Investigation** for use in the eventual focused→global handoff. ### Key finding: Contribution ≠ Finding ```text Contribution: immutable provenance from one answer interpretation. Captures: what the model observed about what the user said at one turn. Contains: observations, uncertainties, assumptions, relationships, follow-ups. Mutated: NO — it is a record of one interpretation event. Finding: living conceptual proposition/theme in the investigation. Accumulates support from multiple contributions across turns and branches. Carries user disposition (agreed / challenged / not relevant). Can change standing; can be reopened by new evidence. ``` ### Why contributions alone are insufficient Evidence from live walkthrough: - **Same idea across turns:** Same conceptual themes emerge in different wording across multiple focused deconstruction turns. Without higher-level grouping, these appear as disconnected observations. - **Latest-result replacement:** `focusedInvestigations[nodeId].result` only holds the latest result. Older findings disappear even when still conceptually valid. - **Cross-branch evidence:** A previous finding may gain or lose significance when new evidence appears on a different branch. The turn-based model has no mechanism to express this cross-branch linkage. - **User validation gap:** Users need lightweight disposition controls (Agree / Not quite / Not relevant). These have no semantic home under a contribution-only model. - **Reopening needed:** A previously "not relevant" finding may become material again when later evidence connects to it. No durable entity exists to carry this state across turns. ### Truth vs relevance must be separate dimensions ```text true but irrelevant — supported by evidence, does not drive current investigation relevant but uncertain — important question with limited supporting evidence agreed and highly relevant — user confirmed; strong global reasoning input challenged but still important — imprecise but guides investigation direction previously irrelevant, reopened by new evidence — status changed, weight restored ``` These cannot be combined into a single score without losing critical investigation state. ### User dispositions are semantic, not UI-only - **Agree** → user treats as working premise (not absolute proof) - **Not quite** → interpretation needs correction (partially supported) - **Not relevant** → may be true but should not drive this investigation (discounted from active reasoning weight, preserved in provenance) - **Silence remains provisional** — never interpret lack of response as agreement These dispositions carry semantic weight that the reasoning system should consume for confidence and handoff decisions. ### Reopening principle > A finding should be durable, but its standing and relevance should remain revisable when new evidence appears elsewhere. Reopening = new evidence has made a previously settled / discounted / irrelevant proposition material to the investigation again. The previous disposition is preserved in provenance; the finding gains new weight with explicit status indicating why it was reopened. ### Contribution → Finding relationship - **Cardinality:** Many-to-one (multiple contributions converge on one evolving conceptual finding) - **Provenance:** Essential — every finding must track which contributions support it, which dispositions were applied, and what later evidence supports or challenges it ### SituationGraph relationship A Finding is a **user-facing grouping** over one or more graph propositions. It is not simply another name for an existing graph node, nor is it unnecessary — it is the conceptual layer the user reasons about, built on top of graph nodes as structural anchors. ### Impact on focused → global handoff **Material change:** The handoff question shifts from "copy raw focused fields into global reasoning" to: ```text focused contribution → candidate / evolving finding → user disposition + provenance + later evidence → finding standing changes → eligible global reasoning input ``` **Blindly converting observations/uncertainties/assumptions/relationships into graph mutations would be unsafe because:** - Assumptions may be wrong (model misattribution) - Relationships may overreach (co-mention ≠ causation) - Relevance may change across investigation branches - Tentative language may have been strengthened in a later turn - Findings may later be reopened — their status is not final at handoff time ### Global eligibility matrix | Item | Eligibility | |------|-------------| | Raw contribution | LOCAL/PROVENANCE-ONLY (unless no finding exists) | | Provisional finding | GLOBAL-ELIGIBLE-WITH-STATUS (reduced weight) | | Agreed finding | GLOBAL-ELIGIBLE (strong support) | | Challenged / not-quite | GLOBAL-ELIGIBLE-WITH-STATUS (partial support) | | Not-relevant finding | LOCAL/PROVENANCE-ONLY (discounted, preserved) | | Reopened finding | GLOBAL-ELIGIBLE-WITH-STATUS (weight restored by new evidence) | | Superseded finding | GLOBAL-ELIGIBLE-WITH-STATUS (use superseding finding) | | Unresolved finding | GLOBAL-ELIGIBLE-WITH-STATUS (triggers further investigation) | ### Primary user-facing reasoning object **Findings**, not contributions or turn history. Contributions and turns are provenance underneath; findings are what the user actually reasons about. The investigation is conceptually about building understanding, not collecting answer-interpreters. ### Product principle alignment > "The Engine's primary job is to progressively build a trustworthy representation of the situation. Questions are the mechanism used to improve that representation." **STRONGLY-ALIGNS.** Confirmed by both the Return-to-Origin methodology and live walkthrough evidence. The engine's product is the trustworthy representation (findings + confidence), not the graph structure or question sequence. ### Confidence emerges from: - Confirmed findings (agreed) - Provisional findings (accumulating support) - Challenged findings (partial, needs refinement) - Irrelevant/discounted findings (understood, not driving) - Contradictions between findings - Unresolved gaps - Evidence provenance (how many contributions support each finding) - Stability under new evidence - Ability to reopen findings when relevance changes ### Decision: FINDINGS-DOMAIN-MODEL-JUSTIFIED The distinction is clearly useful enough that focused→global integration should be designed around findings/provenance rather than raw turn outputs. The semantic gap between "what one answer contributed" and "what the investigation has found so far" is real and material.