docs(confidence-engine): record uncertainty-led investigation hypothesis
This commit is contained in:
@@ -550,6 +550,7 @@ These results are documented as **historical experiment evidence**. The v0.61 li
|
||||
|---|---|
|
||||
| Product evolution v0.51–v0.60 | `docs/design-evolution/README.md` (progressive loading) |
|
||||
| Methodology / RTO axioms | `docs/current-working-principles.md` §0 (A1–A12) |
|
||||
| Uncertainty-led initial investigation hypothesis | `docs/uncertainty-led-investigation-hypothesis.md` — **PROPOSED / UNTESTED — no implementation decision made** |
|
||||
| Architecture guardrails | `.claude/architecture-guardrails.md` |
|
||||
| Task routing by work type | `docs/task-context-packs.md` |
|
||||
| September 8 product/economics checkpoint | `docs/confidence-engine-product-checkpoint-2026-09-08.md` |
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
# Confidence Engine — Uncertainty-Led Investigation Hypothesis
|
||||
|
||||
**Status:** PROPOSED / UNTESTED — no implementation decision made
|
||||
|
||||
**Date:** 9 October 2026
|
||||
|
||||
**Purpose:** Durable project-context checkpoint / Build → Break → Learn → STOP
|
||||
|
||||
## Purpose and status
|
||||
|
||||
This is a durable methodology hypothesis, not an approved architecture or product requirement. It records a possible simplification of Confidence Engine before further optimisation of initial Open Question wording, preserving the reasoning, evidence, limitations, competing explanations, falsification conditions, and explicit decision boundary.
|
||||
|
||||
## Origin: what prompted the hypothesis
|
||||
|
||||
Confidence Engine currently identifies canonical uncertainties during reconstruction and also generates natural-language Open Questions intended to express those uncertainties. The SituationGraph carries these representations into the user experience.
|
||||
|
||||
HS-01 observed one partially faithful question out of three: a future customer-commitment sufficiency uncertainty was narrowed to whether written commitments already existed. After a v0.5 prompt-contract refinement, HS-02 observed three faithful questions in one repeat of the same manufacturing case. That single run does not establish causality or general reliability.
|
||||
|
||||
HS-03 tested four scenarios: six faithful and four partially faithful transformations across ten uncertainties, with all four partial cases in HS-03C (a nine-month service launch with interacting recruitment, integration, and customer-commitment conditions). The generated questions tended to move from underlying feasibility, timing, or dependency uncertainties to present evidence, schedules, or bottlenecks. HS-03C also exposed a separate possible deconstruction omission: an overarching launch-feasibility uncertainty was not surfaced as its own canonical Open Question.
|
||||
|
||||
These results support investigating the framing boundary. They do not prove that question wording is the only problem, that canonical descriptions are always adequate, or that the proposed interaction is superior.
|
||||
|
||||
## The hypothesis
|
||||
|
||||
Confidence Engine may not need to convert each initially identified canonical uncertainty into a question. It may be more faithful and useful to present uncertainties as uncertainties, allowing the user to choose which one to explore.
|
||||
|
||||
The initial stage answers: “What remains uncertain?” The focused-investigation stage asks: “What can we explore to understand it better?” Contextual, evidence-seeking questions belong naturally inside the latter.
|
||||
|
||||
An uncertainty can remain unresolved throughout investigation. The user’s Current Understanding can develop, and the nature or degree of uncertainty can decrease, increase, or change. The user’s confidence in an outcome is not the same as confidence in understanding the situation; confidence, sufficiency, readiness to act, and action remain the user’s judgement.
|
||||
|
||||
## Illustrative interaction, not a specification
|
||||
|
||||
Initial presentation: “We have identified three areas of uncertainty in your situation. You can explore whichever you find useful.” Each item displays its canonical uncertainty with enough context to recognise its meaning; no generated interrogative is required.
|
||||
|
||||
Example uncertainty: “Timeline and feasibility of recruiting two specialists within the nine-month window.”
|
||||
|
||||
After the user selects it, focused investigation may ask: “What recruitment activity has already taken place?” or “What is known about the availability of suitable specialists?” Such questions may help investigate feasibility, but they are not equivalent replacements for the original uncertainty.
|
||||
|
||||
The user can answer questions, pause with “Done for now”, review an updated Current Understanding, defer the uncertainty, and revisit it. There is no mandated priority, compulsory resolution, or confidence score.
|
||||
|
||||
## Why this might simplify the system
|
||||
|
||||
It could remove the canonical-description → initial-question semantic transformation where HS-01 and HS-03C exhibited drift. That may reduce the need for further prompt tightening, fixed dictionaries, dynamic semantic dictionaries, or a separate semantic-equivalence layer for that transformation.
|
||||
|
||||
It does not remove the need for LLM interpretation during deconstruction or contextual reasoning during focused investigation. It also does not guarantee that the original uncertainty was correctly identified or well expressed.
|
||||
|
||||
Cross-domain and eventual multilingual flexibility may benefit from not imposing a fixed catalogue of question forms, but that benefit remains a design expectation, not an observed result.
|
||||
|
||||
## Continuity with Return to Origin
|
||||
|
||||
The originating methodology is an investigation process rather than a question-answer service. The user is the investigator; the graph organises and remembers the work; multiple uncertainties may coexist; progress is not synonymous with answering every question.
|
||||
|
||||
This hypothesis does not replace or reverse Return to Origin. It proposes a clearer boundary between identifying uncertainties and investigating them, consistent with user-selected focus, persistent Current Understanding, user-owned confidence, and “Done for now”.
|
||||
|
||||
Preserve existing graph relationships, stable identities, provenance, persistence, focused-investigation flow, and user ownership unless evidence shows a change is necessary.
|
||||
|
||||
## Competing explanations, failure risks, and falsification conditions
|
||||
|
||||
- The current initial question may provide genuine clarity or accessibility that a terse canonical description lacks. If direct uncertainty presentation reduces comprehension or confidence in choosing a focus, the hypothesis is weakened or falsified.
|
||||
- Focused investigation may rely on the generated initial question as input, an identity cue, or a semantic constraint. If removing or bypassing it degrades contextual questions, graph continuity, or Current Understanding, the hypothesis is weakened or falsified.
|
||||
- Some canonical descriptions may drift, omit overarching uncertainties, or use jargon. If direct presentation merely exposes or amplifies those defects, the proposed simplification does not solve the underlying problem.
|
||||
- The difference between a canonical uncertainty and a user-friendly noninterrogative title or description may still require a small linguistic transformation. Any such transformation must preserve meaning and be tested rather than assumed safe.
|
||||
- Changes to uncertainty or confidence over time must not be automatically inferred as numeric metrics or claims of resolution.
|
||||
- If the existing question transformation consistently improves clarity without material semantic loss across representative domains, retaining it may be preferable.
|
||||
|
||||
## Build → Break → Learn → STOP: proposed experiment
|
||||
|
||||
**BUILD** — First perform a zero-live-call, read-only architecture trace: reconstruction fields → SituationGraph → initial uncertainty display → focused-investigation entry → contextual questions → Current Understanding → “Done for now”. Identify every consumer of the initial question field. Do not modify production code in this phase.
|
||||
|
||||
**BUILD** — If the trace supports it, create the smallest reversible prototype that presents the canonical uncertainty directly and starts the existing focused investigation from it. Retain the existing flow and data for comparison; avoid schema migrations or broad refactors.
|
||||
|
||||
**BREAK** — Try to falsify the hypothesis. Test HS-03C, an ambiguous or jargon-heavy uncertainty, a nonbusiness scenario, and a case where the generated initial question arguably improves clarity. Check whether users understand what they are choosing, whether deeper questions remain contextual, and whether graph identity and Current Understanding remain coherent.
|
||||
|
||||
**LEARN** — Compare semantic fidelity, comprehension, investigation continuity, and user control against current behaviour. Record failures and successes, distinguishing observation from interpretation. Decide to adopt, revise, or reject only after evidence review.
|
||||
|
||||
**STOP** — No prompt-contract change, dictionary, semantic agent, UI refactor, architecture refactor, or production implementation is authorised by this document. Stop at the first apparatus failure or material contradiction and record it separately.
|
||||
|
||||
## Decision boundary and open questions
|
||||
|
||||
- Can focused investigation operate directly from canonical uncertainty and graph context without depending on the generated initial question?
|
||||
- Are canonical uncertainty descriptions consistently comprehensible, or do they need a distinct noninterrogative presentation layer?
|
||||
- How should the UI indicate that an uncertainty is being explored, parked, clarified, or still unresolved without equating status with certainty?
|
||||
- What evidence would demonstrate that uncertainty-led presentation improves rather than merely changes the experience?
|
||||
|
||||
**Decision state:** OPEN.
|
||||
|
||||
**Next permitted action:** Read-only repository trace and evidence audit; no live LLM calls and no production edits.
|
||||
|
||||
**Implementation authority:** None. This document must not be treated as an accepted product requirement or architectural decision.
|
||||
|
||||
## Evidence anchors and provenance
|
||||
|
||||
- Return to Origin methodology context, 18 August 2026: investigation rather than Q&A; user chooses investigation focus; graph as organiser and memory; Build → Break → Learn → STOP.
|
||||
- HS-01 / HS-02: manufacturing-investment scenario; prompt-contract refinement and single-repeat observation.
|
||||
- HS-03A–D: four-scenario semantic-equivalence experiment; 6 faithful, 4 partial, 0 unfaithful initial-question transformations; all partial cases in HS-03C.
|
||||
- Current implementation and branch/HEAD details are repository facts to verify at execution time, not facts established by this hypothesis.
|
||||
Reference in New Issue
Block a user