architecture: introduce investigation narrative layer

2026-08-05 15:53:22 +01:00
parent 6eaf0fc246
commit 863a4589b3
3 changed files with 348 additions and 2 deletions
@@ -514,3 +514,22 @@ Three tiers, applied top to bottom:
 - Prefer labels over descriptions when labels are more concise and clear.
 - Omit items too verbose to scan; do not synthesise rewritten claims.
 - Never invent facts absent from the graph.
+
+## Investigation Narrative
+
+The reasoning graph is the machine representation of the investigation.
+
+The investigation narrative is the human representation.
+
+The UI renders projections from the narrative, not directly from the graph.
+
+Principles:
+
+- Users understand investigations, not graphs.
+- The graph is an internal reasoning structure.
+- The narrative is the explanation of current understanding.
+- Every user-facing panel should consume narrative state where possible.
+- Multiple UI layouts may share the same narrative.
+- Narrative should evolve as evidence changes.
+- Narrative must never invent facts absent from the graph.
+- Narrative explains uncertainty rather than exposing graph mechanics.
@@ -647,13 +647,95 @@ Confirmed.
 - Resolved unknowns and assumptions are factual answers to previously unanswered questions — they should appear in the known section with an epistemic label ("Not yet established" / "To be tested") if their status hasn't been explicitly set.
 - The translation adapter is the right place for this work: it is a single deterministic function, testable in isolation, and its output contracts are stable.

+#### Result
+
+Confirmed.
+
+#### What did we learn?
+
+- Semantic filtering significantly improved Version C.
+- The remaining limitations are architectural rather than visual.
+- Graph nodes still do not naturally map to facilitator language.
+- Users think in investigation progress rather than graph structure.
+- Version C proved the need for an intermediate narrative model.
+
 #### Decision

-Keep the semantic projection approach. The facilitator view now routes by meaning, suppresses structural noise, deduplicates observations, and prefers concrete findings. Experiment 13 is closed.
+Keep the semantic projection approach.
+
+Do not continue improving graph projection indefinitely.
+
+Proceed to designing an Investigation Narrative layer. Experiment 13 is closed.

 ---

-## Emerging Direction
+### Experiment 14 — Investigation Narrative Layer
+
+#### Hypothesis
+
+The graph should remain the internal reasoning model.
+
+A separate narrative model should become the presentation model.
+
+The facilitator UI should consume narrative state rather than graph nodes.
+
+#### Questions
+
+- What information belongs in a narrative?
+- What belongs only in the graph?
+- Which narrative elements can be derived deterministically?
+- What should remain hidden?
+- Can every facilitator panel consume the same narrative object?
+
+#### Status
+
+Architectural experiment.
+
+#### Evaluation
+
+Pending.
+
+---
+
+## Emerging Direction — Investigation Narrative
+
+The Confidence Engine architecture is becoming:
+
+User
+
+↓
+
+Facilitated Conversation
+
+↓
+
+Reasoning Graph
+
+↓
+
+Investigation Narrative
+
+↓
+
+Workspace Projection
+
+↓
+
+User
+
+The reasoning graph becomes the machine representation.
+
+The investigation narrative becomes the human representation.
+
+The UI simply renders whichever projection is appropriate.
+
+This is an emerging architectural direction.
+
+It is intentionally recorded before implementation so future experiments remain aligned.
+
+---
+
+## Emerging Direction — Graph as Source of Truth

 The first UX experiments focused on workspace structure.

@@ -698,3 +780,16 @@ Dark mode is intentionally deferred.
 Once the information architecture and visual hierarchy stabilise we will investigate whether an "Investigation Mode" (rather than a conventional dark mode) improves concentration.

 This should be treated as a future UX experiment rather than an accessibility feature.
+
+## Backlog — Investigation Narrative (Potential Future Work)
+
+These are observations, not implementation tasks.
+
+- Narrative adapter
+- Narrative quality heuristics
+- Narrative progression
+- Narrative completion state
+- Narrative confidence wording
+- Narrative testing
+- Narrative localisation
+- Multiple narrative projections
@@ -0,0 +1,232 @@
+# Investigation Narrative — Architecture Design
+
+> This is a design document only. Do not implement yet.
+
+---
+
+## Purpose
+
+The reasoning graph contains everything the engine knows, why it knows it, and how it connects to other knowledge. It is an excellent internal reasoning model but a poor presentation model for end users.
+
+This document proposes an intermediate architectural layer — the **Investigation Narrative** — that sits between the reasoning graph and the UI.
+
+The narrative translates machine structure into human understanding without altering the reasoning engine, the graph schema, or any external contract.
+
+---
+
+## Architecture
+
+```
+User
+  ↓↑
+Facilitator UI (workspace projection)
+  ↓↑
+Investigation Narrative (presentation model)
+  ↓↑
+Reasoning Graph (reasoning model)
+  ↓↑
+LLM / Ollama / Reasoning Engine
+  ↓↑
+User
+```
+
+### Why this layer is needed
+
+- The graph's nodes and edges describe *how the engine knows*. Users need to understand *what is known* and *what remains uncertain*.
+- Multiple UI projections (facilitator view, developer details, investigation map) can share a single narrative without each re-implementing its own translation of the graph.
+- The narrative can evolve independently of both the graph schema and the UI layout.
+
+---
+
+## Proposed Narrative Structure
+
+Each section describes:
+
+- **Purpose** — why it exists in the narrative
+- **Source** — where it comes from in the graph
+- **Deterministic** — whether it is derived algorithmically or requires judgment
+- **Audience** — who consumes it
+
+### Current Question
+
+- **Purpose**: Tell the user what to think about next.
+- **Source**: The highest-priority unresolved unknown or assumption node in the graph.
+- **Deterministic**: Yes, if priority rules are fixed.
+- **Audience**: The user (primary).
+
+### Current Understanding
+
+- **Purpose**: Summarise what is known so far.
+- **Source**: Resolved nodes and confirmed observations from the graph, filtered for relevance.
+- **Deterministic**: Yes.
+- **Audience**: The user.
+
+### Known Facts
+
+- **Purpose**: List established findings.
+- **Source**: Graph nodes with status "resolved" that are not scaffolding or procedural.
+- **Deterministic**: Yes.
+- **Audience**: The user; also consumed by other narrative sections.
+
+### Active Unknowns
+
+- **Purpose**: Show what is still being investigated and why it matters.
+- **Source**: Graph nodes with status "unknown" or "assumption" that have not been resolved.
+- **Deterministic**: Yes, with pruning for relevance.
+- **Audience**: The user.
+
+### Current Line of Enquiry
+
+- **Purpose**: Explain what the investigation is focusing on right now.
+- **Source**: Connected subgraph around the active unknown — parent nodes and adjacent reasoning paths.
+- **Deterministic**: Yes, if path selection rules are fixed.
+- **Audience**: The user (context).
+
+### Possible Explanations
+
+- **Purpose**: Present alternative hypotheses without asserting any as true.
+- **Source**: Assumption nodes and explanation-type nodes that have not been confirmed.
+- **Deterministic**: Yes, with epistemic labels.
+- **Audience**: The user (evaluating evidence).
+
+### Evidence Gathered
+
+- **Purpose**: Show what the user has contributed and what the engine has discovered.
+- **Source**: Observation nodes in the graph.
+- **Deterministic**: Yes, deduplicated by normalised text.
+- **Audience**: The user (confidence and traceability).
+
+### Confidence Signals
+
+- **Purpose**: Communicate how certain the engine is — without implying false precision.
+- **Source**: Resolution ratio, number of unresolved nodes, depth of supporting paths.
+- **Deterministic**: Yes, as ratios or qualitative descriptors ("partial", "substantial", "limited").
+- **Audience**: The user (calibrating trust).
+
+### Reason Investigation Continues
+
+- **Purpose**: Explain why the engine has not reached a terminal state.
+- **Source**: Active unknowns and their supporting gaps in the graph.
+- **Deterministic**: Yes, derived from unresolved subgraphs.
+- **Audience**: The user (closure and motivation).
+
+### Recent Progress
+
+- **Purpose**: Show what changed since the last turn.
+- **Source**: Nodes whose status changed or whose evidence count changed between states.
+- **Deterministic**: Yes.
+- **Audience**: The user (continuity and momentum).
+
+### Suggested Next Step
+
+- **Purpose**: Give the user a concrete, minimal action.
+- **Source**: The current unknown + what information would resolve it (derived from its parent edges).
+- **Deterministic**: Yes, if suggestion rules are fixed.
+- **Audience**: The user (next interaction).
+
+### Completion Summary
+
+- **Purpose**: Present the final findings when the investigation reaches a terminal state.
+- **Source**: All resolved nodes, reorganised into coherent findings rather than node lists.
+- **Deterministic**: Yes, with curation for coherence.
+- **Audience**: The user (closure and reference).
+
+---
+
+## Narrative Composition Rules
+
+### Never invent facts
+
+Every narrative element must be traceable to one or more graph nodes. No content may appear in the narrative that does not exist somewhere in the reasoning graph.
+
+### Preserve epistemic certainty
+
+If the graph expresses uncertainty, the narrative must express it — not by showing confidence percentages but by using language like "possibly", "to be tested", "not yet established".
+
+### Explain, do not expose
+
+The narrative should answer: *what does this mean?* rather than *how was this computed?* Graph mechanics (node IDs, edge types, traversal depth) belong in Developer Details, not in the user-facing narrative.
+
+### Shared source of truth
+
+A single narrative object should be produced from the graph and consumed by all UI panels. Panels should not reimplement their own derivation logic.
+
+### State-aware framing
+
+The same narrative fields are always available, but their labels and emphasis change based on investigation phase:
+
+- **Early**: "What we know", "What we need to understand"
+- **Active**: "Current understanding", "Still investigating", "Next question"
+- **Terminal**: "Findings", "What the evidence supports", "Remaining uncertainty"
+
+---
+
+## Layer Responsibilities
+
+### Reasoning Graph
+
+**Responsible for:**
+
+- reasoning
+- evidence
+- relationships
+- uncertainty
+- provenance
+- machine state
+
+**Not responsible for:**
+
+- storytelling
+- explanation
+- user wording
+
+### Investigation Narrative
+
+**Responsible for:**
+
+- translating graph meaning
+- selecting relevant information
+- organising investigation state
+- communicating progress
+- preserving epistemic certainty
+
+**Not responsible for:**
+
+- reasoning
+- inference
+- evidence generation
+
+### UI (Facilitator Panels)
+
+**Responsible for:**
+
+- presentation
+- interaction
+- accessibility
+- cognitive load
+
+**Not responsible for:**
+
+- deciding meaning
+- interpreting graph nodes
+
+---
+
+## Emergent Architecture
+
+The Confidence Engine architecture is becoming:
+
+1. **User** — observes, thinks, responds
+2. **Facilitated Conversation** — the active question-response loop
+3. **Reasoning Graph** — machine representation of all knowledge and uncertainty
+4. **Investigation Narrative** — human representation of current understanding
+5. **Workspace Projection** — UI panels rendering the narrative
+6. **User** — reads, evaluates, contributes evidence
+
+The reasoning graph is the machine representation.
+
+The investigation narrative is the human representation.
+
+The UI simply renders whichever projection is appropriate.
+
+This is an emerging architectural direction. It is intentionally recorded before implementation so future experiments remain aligned.