confidence-engine/docs/uncertainty-led-investigation-hypothesis.md

Confidence Engine — Uncertainty-Led Investigation Hypothesis

Status: PROPOSED / UNTESTED — no implementation decision made

Date: 9 October 2026

Purpose: Durable project-context checkpoint / Build → Break → Learn → STOP

Purpose and status

This is a durable methodology hypothesis, not an approved architecture or product requirement. It records a possible simplification of Confidence Engine before further optimisation of initial Open Question wording, preserving the reasoning, evidence, limitations, competing explanations, falsification conditions, and explicit decision boundary.

Origin: what prompted the hypothesis

Confidence Engine currently identifies canonical uncertainties during reconstruction and also generates natural-language Open Questions intended to express those uncertainties. The SituationGraph carries these representations into the user experience.

HS-01 observed one partially faithful question out of three: a future customer-commitment sufficiency uncertainty was narrowed to whether written commitments already existed. After a v0.5 prompt-contract refinement, HS-02 observed three faithful questions in one repeat of the same manufacturing case. That single run does not establish causality or general reliability.

HS-03 tested four scenarios: six faithful and four partially faithful transformations across ten uncertainties, with all four partial cases in HS-03C (a nine-month service launch with interacting recruitment, integration, and customer-commitment conditions). The generated questions tended to move from underlying feasibility, timing, or dependency uncertainties to present evidence, schedules, or bottlenecks. HS-03C also exposed a separate possible deconstruction omission: an overarching launch-feasibility uncertainty was not surfaced as its own canonical Open Question.

These results support investigating the framing boundary. They do not prove that question wording is the only problem, that canonical descriptions are always adequate, or that the proposed interaction is superior.

The hypothesis

Confidence Engine may not need to convert each initially identified canonical uncertainty into a question. It may be more faithful and useful to present uncertainties as uncertainties, allowing the user to choose which one to explore.

The initial stage answers: “What remains uncertain?” The focused-investigation stage asks: “What can we explore to understand it better?” Contextual, evidence-seeking questions belong naturally inside the latter.

An uncertainty can remain unresolved throughout investigation. The user’s Current Understanding can develop, and the nature or degree of uncertainty can decrease, increase, or change. The user’s confidence in an outcome is not the same as confidence in understanding the situation; confidence, sufficiency, readiness to act, and action remain the user’s judgement.

Illustrative interaction, not a specification

Initial presentation: “We have identified three areas of uncertainty in your situation. You can explore whichever you find useful.” Each item displays its canonical uncertainty with enough context to recognise its meaning; no generated interrogative is required.

Example uncertainty: “Timeline and feasibility of recruiting two specialists within the nine-month window.”

After the user selects it, focused investigation may ask: “What recruitment activity has already taken place?” or “What is known about the availability of suitable specialists?” Such questions may help investigate feasibility, but they are not equivalent replacements for the original uncertainty.

The user can answer questions, pause with “Done for now”, review an updated Current Understanding, defer the uncertainty, and revisit it. There is no mandated priority, compulsory resolution, or confidence score.

Why this might simplify the system

It could remove the canonical-description → initial-question semantic transformation where HS-01 and HS-03C exhibited drift. That may reduce the need for further prompt tightening, fixed dictionaries, dynamic semantic dictionaries, or a separate semantic-equivalence layer for that transformation.

It does not remove the need for LLM interpretation during deconstruction or contextual reasoning during focused investigation. It also does not guarantee that the original uncertainty was correctly identified or well expressed.

Cross-domain and eventual multilingual flexibility may benefit from not imposing a fixed catalogue of question forms, but that benefit remains a design expectation, not an observed result.

Continuity with Return to Origin

The originating methodology is an investigation process rather than a question-answer service. The user is the investigator; the graph organises and remembers the work; multiple uncertainties may coexist; progress is not synonymous with answering every question.

This hypothesis does not replace or reverse Return to Origin. It proposes a clearer boundary between identifying uncertainties and investigating them, consistent with user-selected focus, persistent Current Understanding, user-owned confidence, and “Done for now”.

Preserve existing graph relationships, stable identities, provenance, persistence, focused-investigation flow, and user ownership unless evidence shows a change is necessary.

Competing explanations, failure risks, and falsification conditions

• The current initial question may provide genuine clarity or accessibility that a terse canonical description lacks. If direct uncertainty presentation reduces comprehension or confidence in choosing a focus, the hypothesis is weakened or falsified.
• Focused investigation may rely on the generated initial question as input, an identity cue, or a semantic constraint. If removing or bypassing it degrades contextual questions, graph continuity, or Current Understanding, the hypothesis is weakened or falsified.
• Some canonical descriptions may drift, omit overarching uncertainties, or use jargon. If direct presentation merely exposes or amplifies those defects, the proposed simplification does not solve the underlying problem.
• The difference between a canonical uncertainty and a user-friendly noninterrogative title or description may still require a small linguistic transformation. Any such transformation must preserve meaning and be tested rather than assumed safe.
• Changes to uncertainty or confidence over time must not be automatically inferred as numeric metrics or claims of resolution.
• If the existing question transformation consistently improves clarity without material semantic loss across representative domains, retaining it may be preferable.

Build → Break → Learn → STOP: proposed experiment

BUILD — First perform a zero-live-call, read-only architecture trace: reconstruction fields → SituationGraph → initial uncertainty display → focused-investigation entry → contextual questions → Current Understanding → “Done for now”. Identify every consumer of the initial question field. Do not modify production code in this phase.

BUILD — If the trace supports it, create the smallest reversible prototype that presents the canonical uncertainty directly and starts the existing focused investigation from it. Retain the existing flow and data for comparison; avoid schema migrations or broad refactors.

BREAK — Try to falsify the hypothesis. Test HS-03C, an ambiguous or jargon-heavy uncertainty, a nonbusiness scenario, and a case where the generated initial question arguably improves clarity. Check whether users understand what they are choosing, whether deeper questions remain contextual, and whether graph identity and Current Understanding remain coherent.

LEARN — Compare semantic fidelity, comprehension, investigation continuity, and user control against current behaviour. Record failures and successes, distinguishing observation from interpretation. Decide to adopt, revise, or reject only after evidence review.

STOP — No prompt-contract change, dictionary, semantic agent, UI refactor, architecture refactor, or production implementation is authorised by this document. Stop at the first apparatus failure or material contradiction and record it separately.

Decision boundary and open questions

• Can focused investigation operate directly from canonical uncertainty and graph context without depending on the generated initial question?
• Are canonical uncertainty descriptions consistently comprehensible, or do they need a distinct noninterrogative presentation layer?
• How should the UI indicate that an uncertainty is being explored, parked, clarified, or still unresolved without equating status with certainty?
• What evidence would demonstrate that uncertainty-led presentation improves rather than merely changes the experience?

Decision state: OPEN.

Next permitted action: Read-only repository trace and evidence audit; no live LLM calls and no production edits.

Implementation authority: None. This document must not be treated as an accepted product requirement or architectural decision.

Evidence anchors and provenance

• Return to Origin methodology context, 18 August 2026: investigation rather than Q&A; user chooses investigation focus; graph as organiser and memory; Build → Break → Learn → STOP.
• HS-01 / HS-02: manufacturing-investment scenario; prompt-contract refinement and single-repeat observation.
• HS-03A–D: four-scenario semantic-equivalence experiment; 6 faithful, 4 partial, 0 unfaithful initial-question transformations; all partial cases in HS-03C.
• Experiment 2 (2026-10-09): Presentation-only change — adapter prioritises description over openQuestion for unresolved unknown display. Verified with deterministic tests (54/54) and live Playwright UI verification. Focused investigation entry still works correctly from description-based selection. openQuestion generation remains untouched. Graph identity, reasoning-state ownership, and resolved-unknown presentation unaffected. Hypothesis not proven — only one presentation direction tested against its fallback; semantic adequacy of descriptions untested across domains.
• Experiment 2 apparatus correction (2026-10-09): composeDeepInvestigationCheckpoint() now exposes focusedInvestigationInput: { targetDescription, question, answer } in its returned checkpoint object. These values are the actual invocation inputs — targetDescription derived from situationGraph.nodes.find(n => n.id === targetNodeId)?.description, question is previousQuestion ?? "" (NOT node.openQuestion), answer is the function parameter. Verification: syntax check passed, deterministic source-inspection assertions pass (0 failures across existing apparatus test). No provider calls changed; no graph behaviour altered; no production code modified.

Decision state: OPEN (after iteration 6 evidence).

• Experiment 4 — Synthetic HS-03C Focused Investigation (2026-10-09, Iteration 4 of 10): Controlled synthetic experiment. Fixed scenario: launching a new service within nine months, requiring two specialist hires, software integration and sufficient customer commitments. Target node n2yzwlv: "Timeline and feasibility of recruiting two specialists within the nine-month window." Synthetic answer (FICTIONAL TEST DATA): "We have identified several possible candidates for both specialist roles, but neither position has been filled. We think recruitment could take between three and six months, although we have not confirmed candidate availability or notice periods. Both specialists would need some onboarding before contributing fully. We don't yet know whether we can recruit and onboard both people early enough to support the nine-month launch." The generated question was supplied explicitly as previousQuestion via composeDeepInvestigationCheckpoint(). Model: qwen-claude:latest (Ollama). Exactly one live LLM call. Result: PARTIAL semantic fidelity. The model correctly preserved the core uncertainty about whether recruitment can be completed within the nine-month window and correctly distinguished estimates from confirmed timing, possible candidates from secured hires, and expected onboarding from completed onboarding. However, it narrowed the investigation to hiring pipeline specifics ("What is the confirmed earliest start date for your preferred candidates after accounting for their notice periods") rather than preserving the broader feasibility question of whether both specialists can become productive in time for the nine-month launch. The strongest counterevidence: the canonical description "Timeline and feasibility" was reduced to essentially a recruitment logistics investigation, losing the broader cross-area dependency and cascading-delay uncertainty captured in the original HS-03C graph (unexplainedTransitions with ID utrans_dependency). This is consistent with the HS-03C partial pattern from prior experiments. Input provenance: synthetic answer deliberately crafted for this controlled experiment, not a real user response. Graph constructed from HS-03C reconstruction fixture (.evidence-temp/headless-semantic/HS-03C-result.json).

• Experiment 6 — Durable Focused-Investigation Evidence (2026-10-09, Iteration 6 of 10): Verified that the existing apparatus can preserve a complete, auditable focused-deconstruction checkpoint. Added serializeFocusedInvestigationEvidence() to the canonical helper (tests/graph/live-update-experiment-helper.cjs). Pure function accepts a completed checkpoint + identity/metadata params and returns JSON-serializable evidence record: (1) five deconstruction arrays preserved verbatim — observations, uncertainties, assumptions, relationships, possibleFollowUpQuestions; (2) exact inputs — canonicalTargetDescription, actualQuestion, exactAnswer; (3) execution metadata — null for fields not exposed by the apparatus. Deterministic verification: syntax check passed, 79/79 assertions passed including 28 new evidence-ser tests. No production code modified. No live LLM calls. No new semantic evidence produced. Experiment 4's missing structured output still cannot be reconstructed (never persisted by the apparatus at that time). Evidence-capture limitation corrected for future experiments.

• Experiment 7 — Durable Focused-Investigation Disk Persistence (2026-10-09, Iteration 7 of 10): Verified that serializeFocusedInvestigationEvidence() output survives write-to-disk and read-back without loss. Extended deterministic test in tests/graph/live-update-experiment-helper-checkpoint.test.cjs writes serialized evidence to OS temp directory via os.tmpdir(), reads it back, parses recovered JSON, and asserts deep structural equality against the original. Explicitly verified: five structured arrays with correct lengths; exact synthetic inputs (canonicalTargetDescription, actualQuestion, exactAnswer) unchanged; identity fields preserved; execution metadata null-fields intact. Temp file removed in finally block. Zero live model calls. Zero browser calls. No production or canonical-helper modifications. Total test result: all assertions passed (0 failures). This test proves the serializer's JSON-serializable output survives disk persistence at the apparatus level — it does not establish a production/live-run command for writing evidence to files, which remains an untested operational boundary. Experiment 4's original structured output remains unavailable; never persisted by the apparatus at that time.

• Experiment 8 — Explicit Evidence Persistence Operation (2026-10-09, Iteration 8 of 10): Added saveFocusedInvestigationEvidence(checkpoint, identity, metadata, outputPath) to the canonical helper (tests/graph/live-update-experiment-helper.cjs). This is a small wrapper around serializeFocusedInvestigationEvidence() that performs explicit persistence via an explicitly supplied output path. Validates: (1) non-empty output path required; (2) exactly one valid deconstruction contribution in checkpoint (rejects ambiguous multi-candidate); (3) all four identity fields present (no nulls); (4) focused-investigation input values present. Writes formatted JSON, reads back, verifies deep structural equality. Returns { output_path, verifiedEvidence }. Zero live model calls. Zero browser calls. No production code modified. Total deterministic test result: 0 failures across all assertions including positive tests (file write, structured evidence completeness, input fidelity, identity fidelity, return values) and negative tests (missing path rejection, missing contribution rejection, ambiguous contribution rejection). All OS temp files cleaned in finally blocks. Future live experiments must explicitly call this save function after a completed checkpoint. Experiment 4's original structured output remains unavailable; never persisted by the apparatus at that time.

• Iteration 3 — Extract Focused Target Input Deterministic Test (Recovery): Purified focused-investigation target-node extraction into a standalone pure function extractFocusedTargetInput(situationGraph, targetNodeId) within tests/graph/live-update-experiment-helper.cjs. The function uses situationGraph.nodes.find(n => n.id === targetNodeId) with ?? "" empty-string fallback for unknown nodes. Reused inside the guarded focused-deconstruction path of composeDeepInvestigationCheckpoint. Deterministically tested with known-node exact-match assertions, unknown-node empty-fallback assertions, graph-immutable-side-effect assertion, and source-pattern integration assertion — all 5 assertions PASS. Zero live model calls.

• Experiment 9 — Explicit Dependency Evidence and Focused Investigation (2026-10-09, Iteration 9 of 10): Single-live-run semantic observation. Controlled scenario: synthetic answer explicitly raises three cross-area dependencies (recruitment → software integration dependency, integration → customer commitments dependency, cascading-delay risk). Model response: PARTIAL fidelity. Key finding — the model represented two explicit cross-area relationships and retained the overarching feasibility uncertainty ("Whether unconfirmed candidate notice periods and required onboarding duration leave enough time for software integration and customer commitments within nine months"), which was notably absent in Experiment 4's PARTIAL result. The follow-up question classifies as LEGITIMATE SPECIFICITY (narrows to notice periods but retains broader timeline context). Strongest counterevidence: overarching uncertainty combines multiple domains into one compound question, potentially obscuring the original recruitment-specific framing. Follow-up focuses on hiring logistics with cross-area context rather than independently investigating each dependency layer. Single-run limitation. Evidence file: .evidence-temp/headless-semantic/HS-03C-focused-E9.json.

• CE-ULI-08 — Apparatus for Condition A/B Uncertainty Comparison (Iteration 3 of 10): Bounded experimental apparatus implementing a two-condition treatment to test whether removing the question field from importantUnknowns in reconstruction output improves semantic fidelity. Changes: (1) lib/reconstruction/schema-variant.js — new conditionBJsonSchema using itemSchemaV1 (id, description, confidence only; no question) and exports production conditionAJsonSchema; (2) prompts/reconstruct-v0.5-b-noreq.md — alt prompt with identical line count to production reconstruct-v0.5.md, Rule 7a rewritten to use description-only uncertainty representation instead of interrogative form, no question-generation language; (3) lib/graph/orchestrator.js — passes generationSchema through dependencies when present; (4) lib/analysis.js — accepts opts.generationSchema ?? reconstructionGenerationJsonSchema for experimental override; (5) scripts/start-case-experiment-helper.cjs — --condition B/A flag loads alt prompt/schema or falls back to production defaults; (6) tests/scripts/start-case-experiment-helper.test.js — 8 new tests verifying schema structure, prompt differences, and seam passthrough. 43 targeted deterministic tests PASS. Full suite was interrupted and is not evidence of a complete suite pass. No live model calls. Production callers still use the original reconstruct-v0.5.md prompt and production schema unchanged (fallback path via ??). Both conditions use the canonical provider invocation seam.

• CE-ULI-09 — First Live Comparison: Question-Required vs Question-Free Reconstruction (2026-10-12, Iteration 4 of 10): One live run per condition using fixed HS-03C input (new-service-launch). Model: qwen-claude:latest via Ollama (http://192.168.1.111:11434). Condition A (question-required): success, validation valid, 1 important unknown with description + question, end-to-end 145,668 ms (model response 88,933 ms). Condition B (question-free): success, validation valid (partial — nonfatal question_missing warning), 1 important unknown with description only (no question field), end-to-end 156,064 ms (model response 107,225 ms). Both conditions identified the same material uncertainty: feasibility of completing recruitment, software integration, and customer commitments within nine months given their interdependencies. Uncertainty preserved as unresolved with medium confidence. Neither introduced unsupported premises or premature narrowing. Condition A's description included "potential" qualifier on interdependencies; Condition B used "given their interdependencies" (definitive framing — minor wording difference, no semantic impact). Condition A produced 4 explicit relationships showing each requirement's potential to affect launch feasibility; Condition B produced 2 relationships with different structure but equivalent coverage. Graph topology: both 8 nodes, 5 edges (different node/edge IDs due to independent generation). Timing: Condition B ~10.4% slower end-to-end and ~20.5% slower model response — within normal variation range, not attributed to structural difference without further evidence. Conclusion: Question-free reconstruction produces a valid result through the canonical pipeline with semantically equivalent uncertainty descriptions for this scenario shape. Does not establish superiority of either condition. The hypothesis remains untested in terms of comprehension or investigation quality — only viability of the question-free path was established. Single paired observation, single scenario, single model run per condition. Evidence files: .evidence-temp/headless-semantic/HS-03C-CE-ULI-09-conditionA.json, .evidence-temp/headless-semantic/HS-03C-CE-ULI-09-conditionB.json.

• CE-ULI-10 — Paired Uncertainty-Description Comparison Without Question Generation (2026-10-12, Iteration 5 of 10): Two live calls per condition using fixed HS-03C input. Model: qwen-claude:latest via Ollama. Condition A (question-required): 2 importantUnknowns (broad feasibility + cascading delays), 8 nodes, validation valid, end-to-end 156,572 ms. Condition B (question-free): 1 importantUnknown (combined timelines and execution order), 7 nodes, validation partial (nonfatal question_missing), end-to-end 126,490 ms. Finding: did not support the hypothesis. Both conditions narrowed through different mechanisms — A kept descriptions broad but generated narrowing questions; B avoided question generation but compressed uncertainty structurally into a single compound description. Neither preserved the original four-domain structure of HS-03C. Crucial asymmetry: absence of a follow-up question (Condition B) did NOT mean preservation of broader uncertainty. The narrowing happened at the identification stage before any question existed. Hypothesis status: unconfirmed for this scenario shape. Further multi-scenario comparison needed. Evidence file: .evidence-temp/headless-semantic/HS-03C-CE-ULI-10-conditionA.json (Condition B evidence captured inline — same path pattern).