confidence-engine/docs/uncertainty-led-investigation-hypothesis.md

Confidence Engine — Uncertainty-Led Investigation Hypothesis

Status: PROPOSED / UNTESTED — no implementation decision made

Date: 9 October 2026

Purpose: Durable project-context checkpoint / Build → Break → Learn → STOP

Purpose and status

This is a durable methodology hypothesis, not an approved architecture or product requirement. It records a possible simplification of Confidence Engine before further optimisation of initial Open Question wording, preserving the reasoning, evidence, limitations, competing explanations, falsification conditions, and explicit decision boundary.

Origin: what prompted the hypothesis

Confidence Engine currently identifies canonical uncertainties during reconstruction and also generates natural-language Open Questions intended to express those uncertainties. The SituationGraph carries these representations into the user experience.

HS-01 observed one partially faithful question out of three: a future customer-commitment sufficiency uncertainty was narrowed to whether written commitments already existed. After a v0.5 prompt-contract refinement, HS-02 observed three faithful questions in one repeat of the same manufacturing case. That single run does not establish causality or general reliability.

HS-03 tested four scenarios: six faithful and four partially faithful transformations across ten uncertainties, with all four partial cases in HS-03C (a nine-month service launch with interacting recruitment, integration, and customer-commitment conditions). The generated questions tended to move from underlying feasibility, timing, or dependency uncertainties to present evidence, schedules, or bottlenecks. HS-03C also exposed a separate possible deconstruction omission: an overarching launch-feasibility uncertainty was not surfaced as its own canonical Open Question.

These results support investigating the framing boundary. They do not prove that question wording is the only problem, that canonical descriptions are always adequate, or that the proposed interaction is superior.

The hypothesis

Confidence Engine may not need to convert each initially identified canonical uncertainty into a question. It may be more faithful and useful to present uncertainties as uncertainties, allowing the user to choose which one to explore.

The initial stage answers: “What remains uncertain?” The focused-investigation stage asks: “What can we explore to understand it better?” Contextual, evidence-seeking questions belong naturally inside the latter.

An uncertainty can remain unresolved throughout investigation. The user’s Current Understanding can develop, and the nature or degree of uncertainty can decrease, increase, or change. The user’s confidence in an outcome is not the same as confidence in understanding the situation; confidence, sufficiency, readiness to act, and action remain the user’s judgement.

Illustrative interaction, not a specification

Initial presentation: “We have identified three areas of uncertainty in your situation. You can explore whichever you find useful.” Each item displays its canonical uncertainty with enough context to recognise its meaning; no generated interrogative is required.

Example uncertainty: “Timeline and feasibility of recruiting two specialists within the nine-month window.”

After the user selects it, focused investigation may ask: “What recruitment activity has already taken place?” or “What is known about the availability of suitable specialists?” Such questions may help investigate feasibility, but they are not equivalent replacements for the original uncertainty.

The user can answer questions, pause with “Done for now”, review an updated Current Understanding, defer the uncertainty, and revisit it. There is no mandated priority, compulsory resolution, or confidence score.

Why this might simplify the system

It could remove the canonical-description → initial-question semantic transformation where HS-01 and HS-03C exhibited drift. That may reduce the need for further prompt tightening, fixed dictionaries, dynamic semantic dictionaries, or a separate semantic-equivalence layer for that transformation.

It does not remove the need for LLM interpretation during deconstruction or contextual reasoning during focused investigation. It also does not guarantee that the original uncertainty was correctly identified or well expressed.

Cross-domain and eventual multilingual flexibility may benefit from not imposing a fixed catalogue of question forms, but that benefit remains a design expectation, not an observed result.

Continuity with Return to Origin

The originating methodology is an investigation process rather than a question-answer service. The user is the investigator; the graph organises and remembers the work; multiple uncertainties may coexist; progress is not synonymous with answering every question.

This hypothesis does not replace or reverse Return to Origin. It proposes a clearer boundary between identifying uncertainties and investigating them, consistent with user-selected focus, persistent Current Understanding, user-owned confidence, and “Done for now”.

Preserve existing graph relationships, stable identities, provenance, persistence, focused-investigation flow, and user ownership unless evidence shows a change is necessary.

Competing explanations, failure risks, and falsification conditions

• The current initial question may provide genuine clarity or accessibility that a terse canonical description lacks. If direct uncertainty presentation reduces comprehension or confidence in choosing a focus, the hypothesis is weakened or falsified.
• Focused investigation may rely on the generated initial question as input, an identity cue, or a semantic constraint. If removing or bypassing it degrades contextual questions, graph continuity, or Current Understanding, the hypothesis is weakened or falsified.
• Some canonical descriptions may drift, omit overarching uncertainties, or use jargon. If direct presentation merely exposes or amplifies those defects, the proposed simplification does not solve the underlying problem.
• The difference between a canonical uncertainty and a user-friendly noninterrogative title or description may still require a small linguistic transformation. Any such transformation must preserve meaning and be tested rather than assumed safe.
• Changes to uncertainty or confidence over time must not be automatically inferred as numeric metrics or claims of resolution.
• If the existing question transformation consistently improves clarity without material semantic loss across representative domains, retaining it may be preferable.

Build → Break → Learn → STOP: proposed experiment

BUILD — First perform a zero-live-call, read-only architecture trace: reconstruction fields → SituationGraph → initial uncertainty display → focused-investigation entry → contextual questions → Current Understanding → “Done for now”. Identify every consumer of the initial question field. Do not modify production code in this phase.

BUILD — If the trace supports it, create the smallest reversible prototype that presents the canonical uncertainty directly and starts the existing focused investigation from it. Retain the existing flow and data for comparison; avoid schema migrations or broad refactors.

BREAK — Try to falsify the hypothesis. Test HS-03C, an ambiguous or jargon-heavy uncertainty, a nonbusiness scenario, and a case where the generated initial question arguably improves clarity. Check whether users understand what they are choosing, whether deeper questions remain contextual, and whether graph identity and Current Understanding remain coherent.

LEARN — Compare semantic fidelity, comprehension, investigation continuity, and user control against current behaviour. Record failures and successes, distinguishing observation from interpretation. Decide to adopt, revise, or reject only after evidence review.

STOP — No prompt-contract change, dictionary, semantic agent, UI refactor, architecture refactor, or production implementation is authorised by this document. Stop at the first apparatus failure or material contradiction and record it separately.

Decision boundary and open questions

• Can focused investigation operate directly from canonical uncertainty and graph context without depending on the generated initial question?
• Are canonical uncertainty descriptions consistently comprehensible, or do they need a distinct noninterrogative presentation layer?
• How should the UI indicate that an uncertainty is being explored, parked, clarified, or still unresolved without equating status with certainty?
• What evidence would demonstrate that uncertainty-led presentation improves rather than merely changes the experience?

Decision state: OPEN.

Next permitted action: Read-only repository trace and evidence audit; no live LLM calls and no production edits.

Implementation authority: None. This document must not be treated as an accepted product requirement or architectural decision.

Evidence anchors and provenance

• Return to Origin methodology context, 18 August 2026: investigation rather than Q&A; user chooses investigation focus; graph as organiser and memory; Build → Break → Learn → STOP.
• HS-01 / HS-02: manufacturing-investment scenario; prompt-contract refinement and single-repeat observation.
• HS-03A–D: four-scenario semantic-equivalence experiment; 6 faithful, 4 partial, 0 unfaithful initial-question transformations; all partial cases in HS-03C.
• Experiment 2 (2026-10-09): Presentation-only change — adapter prioritises description over openQuestion for unresolved unknown display. Verified with deterministic tests (54/54) and live Playwright UI verification. Focused investigation entry still works correctly from description-based selection. openQuestion generation remains untouched. Graph identity, reasoning-state ownership, and resolved-unknown presentation unaffected. Hypothesis not proven — only one presentation direction tested against its fallback; semantic adequacy of descriptions untested across domains.
• Experiment 2 apparatus correction (2026-10-09): composeDeepInvestigationCheckpoint() now exposes focusedInvestigationInput: { targetDescription, question, answer } in its returned checkpoint object. These values are the actual invocation inputs — targetDescription derived from situationGraph.nodes.find(n => n.id === targetNodeId)?.description, question is previousQuestion ?? "" (NOT node.openQuestion), answer is the function parameter. Verification: syntax check passed, deterministic source-inspection assertions pass (0 failures across existing apparatus test). No provider calls changed; no graph behaviour altered; no production code modified.

Decision state: OPEN (after iteration 6 evidence).

• Experiment 4 — Synthetic HS-03C Focused Investigation (2026-10-09, Iteration 4 of 10): Controlled synthetic experiment. Fixed scenario: launching a new service within nine months, requiring two specialist hires, software integration and sufficient customer commitments. Target node n2yzwlv: "Timeline and feasibility of recruiting two specialists within the nine-month window." Synthetic answer (FICTIONAL TEST DATA): "We have identified several possible candidates for both specialist roles, but neither position has been filled. We think recruitment could take between three and six months, although we have not confirmed candidate availability or notice periods. Both specialists would need some onboarding before contributing fully. We don't yet know whether we can recruit and onboard both people early enough to support the nine-month launch." The generated question was supplied explicitly as previousQuestion via composeDeepInvestigationCheckpoint(). Model: qwen-claude:latest (Ollama). Exactly one live LLM call. Result: PARTIAL semantic fidelity. The model correctly preserved the core uncertainty about whether recruitment can be completed within the nine-month window and correctly distinguished estimates from confirmed timing, possible candidates from secured hires, and expected onboarding from completed onboarding. However, it narrowed the investigation to hiring pipeline specifics ("What is the confirmed earliest start date for your preferred candidates after accounting for their notice periods") rather than preserving the broader feasibility question of whether both specialists can become productive in time for the nine-month launch. The strongest counterevidence: the canonical description "Timeline and feasibility" was reduced to essentially a recruitment logistics investigation, losing the broader cross-area dependency and cascading-delay uncertainty captured in the original HS-03C graph (unexplainedTransitions with ID utrans_dependency). This is consistent with the HS-03C partial pattern from prior experiments. Input provenance: synthetic answer deliberately crafted for this controlled experiment, not a real user response. Graph constructed from HS-03C reconstruction fixture (.evidence-temp/headless-semantic/HS-03C-result.json).

• Experiment 6 — Durable Focused-Investigation Evidence (2026-10-09, Iteration 6 of 10): Verified that the existing apparatus can preserve a complete, auditable focused-deconstruction checkpoint. Added serializeFocusedInvestigationEvidence() to the canonical helper (tests/graph/live-update-experiment-helper.cjs). Pure function accepts a completed checkpoint + identity/metadata params and returns JSON-serializable evidence record: (1) five deconstruction arrays preserved verbatim — observations, uncertainties, assumptions, relationships, possibleFollowUpQuestions; (2) exact inputs — canonicalTargetDescription, actualQuestion, exactAnswer; (3) execution metadata — null for fields not exposed by the apparatus. Deterministic verification: syntax check passed, 79/79 assertions passed including 28 new evidence-ser tests. No production code modified. No live LLM calls. No new semantic evidence produced. Experiment 4's missing structured output still cannot be reconstructed (never persisted by the apparatus at that time). Evidence-capture limitation corrected for future experiments.