confidence-engine/docs/investigation-state-assessment.md

# Investigation State Assessment — Architectural Specification

> **Status: Passive / Experimental Implementation**
> Implemented in code during Experiments 18–25B but does not control active user-facing investigation routing.
> The capability records signals for future use; it is not current authority over the investigation path.
> Design evolved through experiments; implementation validates and adjusts the spec iteratively.

## Implementation status

- **Implemented in code**: Yes — first executable slice deployed (Experiment 18).
- **Active routing authority**: No — passive experimental capability isolated from user-facing investigation.

---

## Purpose

The facilitator behaviour model (Experiment 15) defines *what* an expert consultant does during an investigation. It does not define *how the system decides which behaviour to deploy*.

This document introduces an architectural layer that sits between the Investigation Narrative and Behaviour Selection: the **Investigation State Assessment**.

Its purpose is to answer one question before any behaviour is selected:

> Given where the investigation is right now, what kind of help is most appropriate?

The assessment does not decide. It describes. The decision process consumes its output.

---

## Architecture

```
User
  ↓↑
Facilitated Conversation (behaviour deployed here)
  ↓↑
Behaviour Selection (consumes assessment output)
  ↓↑
Investigation State Assessment (evaluates investigation)
  ↓↑
Investigation Narrative (translates graph to human state)
  ↓↑
Reasoning Graph (machine representation)
  ↓↑
LLM / Ollama / Reasoning Engine
  ↓↑
User
```

The assessment layer sits between narrative and behaviour selection.

It reads investigation state from the narrative. It produces a structured evaluation that behaviour selection consumes.

It does not inspect graph nodes directly.

---

## The Assessment Problem

Without an assessment layer, behaviour selection has two flawed options:

1. **Inspect the graph directly.** This couples behaviour to reasoning implementation. Any graph schema change breaks behaviour decisions.
2. **Use narrative fields as proxy signals.** This is fragile because narrative was designed for presentation, not decision-making.

The assessment layer exists so that behaviour selection depends on *investigation state* rather than *implementation details*.

---

## Assessment Dimensions

Each dimension describes what the facilitator should know before acting.

For each: purpose, observable signals, possible values, and how behaviours may consume it.

---

### 1. Current Investigation Phase

**Purpose:** Determine which phase of investigation is active. Phase constrains which behaviours are appropriate.

**Observable signals:**
- How much evidence has been gathered relative to the situation's complexity
- Whether known items are mostly broad or detailed
- Whether active unknowns target foundational or peripheral questions
- The spread and depth of resolved subgraphs in the narrative

**Possible values:**
- `Orienting` — initial understanding being built, nothing established yet
- `Exploring` — breadth phase, multiple directions open
- `Focusing` — patterns emerging, directions narrowing
- `Deepening` — specific areas under scrutiny
- `Synthesising` — understanding coalescing, connections forming
- `Concluding` — sufficient resolution for current scope

**How behaviours consume it:**
- `Orient` is only appropriate in `Orienting` phase
- `Exploring` favours breadth behaviours: Clarify, Observe pattern, Connect
- `Focusing` favours: Decide direction, Progressively narrow focus
- `Deepening` favours: Challenge assumption, Validate, Refine understanding
- `Synthesising` favours: Summarise, Observe pattern, Communicate confidence honestly
- `Concluding` favours: Avoid premature closure, Communicate confidence honestly

---

### 2. Investigation Progress

**Purpose:** Determine whether the investigation is moving forward and at what velocity. Progress state affects pacing and behaviour urgency.

**Observable signals:**
- Rate of new resolved items per turn
- Whether recent turns add fundamentally new information or refine existing understanding
- Whether active unknowns are being resolved or replaced by new ones
- Depth of reasoning paths from conclusions to established facts

**Possible values:**
- `Accelerating` — each turn adds meaningful new resolution
- `Steady` — consistent, predictable progress
- `Stalled` — turns do not advance understanding
- `Looping` — similar questions or responses repeat across turns
- `Spiralling` — investigation widens without deepening in any direction

**How behaviours consume it:**
- `Accelerating` may trigger: Refine understanding, Observe pattern
- `Steady` is stable — continue current behaviour track
- `Stalled` triggers intervention: Challenge assumption, Decide direction, Pause
- `Looping` triggers: Pause, Clarify, or reorient the line of enquiry
- `Spiralling` triggers: Progressively narrow focus, Decide direction

---

### 3. Evidence Quality

**Purpose:** Determine the reliability and coherence of what has been established. Evidence quality determines whether the facilitator should push for more or consolidate.

**Observable signals:**
- Source diversity of evidence (single assertion vs. multiple converging sources)
- Presence of contradictions between established items
- Whether evidence relies on untested assumptions as premises
- Strength of reasoning paths connecting findings to conclusions

**Possible values:**
- `Weak` — little reliable evidence, mostly assertions or single-source information
- `Mixed` — some strong evidence alongside weaker or contested items
- `Strong` — multiple converging sources, coherent narrative, tested assumptions
- `Contradictory` — established items conflict in ways that have not been resolved

**How behaviours consume it:**
- `Weak` triggers: Clarify, Validate (what little exists), Challenge assumption
- `Mixed` triggers: Validate (strong items), Challenge assumption (weak items)
- `Strong` favours: Connect, Observe pattern, Refine understanding, Communicate confidence honestly
- `Contradictory` triggers: Challenge assumption, Expose uncertainty, Clarify

---

### 4. Understanding Trajectory

**Purpose:** Determine whether the investigation's understanding is growing, static, or degrading. This affects whether to push forward or consolidate.

**Observable signals:**
- Number of new coherent insights per turn (not just facts, but meaningful connections)
- Whether new information clarifies previous items or introduces new confusion
- Whether active unknowns are becoming better defined or more vague over turns
- The ratio of synthesized understanding to raw evidence accumulation

**Possible values:**
- `Growing` — each turn produces both facts and meaningful synthesis
- `Static` — information accumulates without deeper understanding forming
- `Confused` — new inputs introduce ambiguity rather than clarity
- `Consolidating` — understanding is stabilizing, fewer net insights but more coherence

**How behaviours consume it:**
- `Growing` favours: Connect, Observe pattern, Progressively narrow focus
- `Static` triggers: Challenge assumption, Decide direction, Pause (ask if user needs prompting)
- `Confused` triggers: Clarify, Expose uncertainty, Pause (hold space for clarity)
- `Consolidating` favours: Refine understanding, Summarise, Communicate confidence honestly

---

### 5. Uncertainty Trend

**Purpose:** Determine whether the investigation's overall uncertainty is increasing, reducing, or stable. This affects pacing and whether to push toward resolution.

**Observable signals:**
- Resolution ratio trending up vs. down across recent turns
- Whether new unknowns are being introduced faster than old ones resolved
- The severity (not just count) of remaining active unknowns
- Proportion of critical-path items still unresolved vs. peripheral items

**Possible values:**
- `Increasing` — more or deeper uncertainties emerging than resolving
- `Reducing` — clear trajectory toward resolution
- `Stable` — uncertainty holding steady, neither improving nor worsening
- `Asymmetric` — some areas well-resolved while others remain deeply uncertain

**How behaviours consume it:**
- `Increasing` triggers: Decide direction (redirect), Challenge assumption, Pause
- `Reducing` favours: Validate, Summarise, Communicate confidence honestly
- `Stable` is neutral — continue current track, monitor for change
- `Asymmetric` triggers: Expose uncertainty, Decide direction

---

### 6. Conversation Health

**Purpose:** Evaluate the quality of the interaction pattern between user and facilitator. Poor conversation health indicates the current approach needs adjustment regardless of investigation state.

**Observable signals:**
- Repetition of question types or response patterns across turns
- User response length trending shorter (disengagement) or longer (confusion, over-explaining)
- Whether the user is answering what is asked or redirecting to different topics
- Frequency of meta-comments ("What are we trying to find out?")

**Possible values:**
- `Healthy` — natural back-and-forth, appropriate depth, engaged responses
- `Repetitive` — similar question-response patterns repeating without progress
- `Too broad` — user responses cover too much ground, losing focus
- `Too narrow` — investigation compressed to one dimension, missing context
- `User overloaded` — user asked to process too much per turn
- `User under-informed` — user lacks sufficient context to give useful answers

**How behaviours consume it:**
- `Healthy` is stable — continue current track
- `Repetitive` triggers: Pause, Clarify, Decide direction (change approach)
- `Too broad` triggers: Progressively narrow focus, Decide direction
- `Too narrow` triggers: Expose uncertainty, Challenge assumption, Connect
- `User overloaded` triggers: Pause, Acknowledge (reduce pressure), Communicate confidence honestly
- `User under-informed` triggers: Clarify, Orient, Expose uncertainty

---

### 7. Behaviour Readiness

**Purpose:** A derived signal — not assessed independently but synthesized from the dimensions above. It answers: *Which behaviours are available and appropriate right now?*

This is the assessment's output layer. Each dimension feeds into behaviour readiness as a set of weighted signals rather than a single verdict.

**Signals produced (not decided):**
- Which behaviours are *available* (phase permits)
- Which behaviours are *pressed for* (multiple dimensions converge on the same intervention)
- Which behaviours are *inappropriate* (contraindicated by current state)
- Which behaviours are *stable* (appropriate across all or most states)

**How behaviours consume it:**
Behaviour selection does not read individual dimensions. It reads behaviour readiness as a single structured signal and selects accordingly. This keeps behaviour independent from assessment implementation details.

The facilitator should understand the readiness layer as a lens — an interpretation of raw state through multiple analytical dimensions — rather than as a decision mechanism itself. The readiness signals inform; they do not decide.

---

## Assessment Principles

### Principle 1: Assess, Do Not Decide

The assessment describes the investigation. It does not select behaviours.

**Purpose:** Separate evaluation from action. A cleaner separation means each layer can evolve independently.

**Observable signals:** N/A — this is an operating constraint on the layer itself.

**How behaviours may consume it:** Behaviours read assessment output as state, not as instructions. The same assessment should support multiple behaviour selection strategies (deterministic rules, weighted scoring, or LLM-assisted). The assessment does not lock in any particular strategy.

---

### Principle 2: All Signals Traceable to Narrative

Every assessment signal must be derivable from the Investigation Narrative without reading the graph directly.

**Purpose:** Keep behaviour decoupled from reasoning implementation. If the graph schema changes, the assessment does not need to change — as long as the narrative preserves its fields.

**Observable signals:** The narrative has all necessary data: phase indicators, resolution counts, evidence quality signals, turn history, response patterns. None of these require graph traversal.

**How behaviours may consume it:** Behaviours trust the assessment as a stable contract. If the assessment accurately reflects investigation state through narrative fields, the behaviour layer remains correct regardless of how the narrative is derived.

---

### Principle 3: Signals Are Descriptive, Not Prescriptive

Each signal describes *what is happening*, not *what should be done*.

**Purpose:** Prevent the assessment from becoming a decision tree in disguise. A descriptive assessment supports multiple interpretation strategies; a prescriptive one locks into one.

**Observable signals:** Each dimension's "possible values" are neutral descriptions. "Stalled" does not mean "use Pause" — it means "progress has stopped." What to do about that is a behaviour selection question.

**How behaviours may consume it:** Different investigation contexts may interpret the same signal differently. `Stalled` in early phase might mean "need more context"; in late phase it might mean "reach closure." The assessment does not encode either interpretation.

---

### Principle 4: Convergence Matters More Than Any Single Signal

A behaviour should be selected when multiple dimensions converge, not when one dimension reaches a threshold.

**Purpose:** Prevent brittle, single-signal triggers that produce inappropriate responses in edge cases. Convergent signals indicate robust state patterns.

**Observable signals:** Cross-dimensional correlation — e.g., `Looping` progress + `Repetitive` conversation + `Confused` understanding is a stronger signal than any one alone.

**How behaviours may consume it:** The behaviour readiness layer should express convergence explicitly: which dimensions agree on which intervention area, and to what degree. Behaviours are more confident when multiple signals align.

---

### Principle 5: Assessment Is Stateful Across Turns

The assessment is not recomputed from scratch each turn. It accumulates and evolves.

**Purpose:** Investigation state is temporal. Current status alone cannot capture looping, acceleration, or convergence. Turn history is essential to meaningful assessment.

**Observable signals:** Turn-level deltas (what changed), sequence patterns (what repeated), trend direction (accelerating, decelerating), phase transitions.

**How behaviours may consume it:** Behaviours that depend on temporal patterns (Looping detection, Progressive narrowing) need the assessment to carry forward state between turns. A turn-by-turn stateless assessment cannot detect these patterns.

---

### Principle 6: Uncertainty About Assessment Is Itself Assessable

The assessment should be able to express its own uncertainty about dimensions it cannot reliably evaluate.

**Purpose:** Avoid false precision. Some investigation states are genuinely hard to classify (e.g., early phase with ambiguous inputs). The assessment should say "I cannot determine this reliably" rather than guessing.

**Observable signals:** Insufficient data, conflicting signals across dimensions, rapid state changes within a single turn, user input that does not map cleanly to existing categories.

**How behaviours may consume it:** Behaviours should handle uncertain assessment fields gracefully — defaulting to conservative, non-committing actions (Acknowledge, Clarify) when the assessment cannot determine direction.

---

## Decision Matrix

An exploratory design aid. Shows which investigation states most naturally lead to which behaviour types. These are patterns, not rules.

| Investigation State | Likely Behaviour | Reason |
|---------------------|-----------------|--------|
| Orienting + stalled | Clarify | Too many unknowns remain; the starting point needs anchoring. |
| Exploring + broad | Observe pattern | Multiple data points suggest a theme before pushing for specifics. |
| Focusing + asymmetric uncertainty | Expose uncertainty | Knowing some areas well while others remain fuzzy demands visibility. |
| Deepening + strong evidence | Validate | Solid findings should be marked and integrated before adding more. |
| Deepening + contradictory evidence | Challenge assumption | Evidence conflicts; the foundation of one or more lines of enquiry is shaky. |
| Synthesising + reducing uncertainty | Refine understanding / Summarise | Shared understanding is coalescing; compress without losing detail. |
| Any phase + looping detected | Pause | Risk of conversational repetition; hold space for reflection. |
| Convergent high evidence quality | Communicate confidence honestly | The state warrants explicit confidence language — match epistemic reality. |
| Low uncertainty + sufficient resolution | Avoid premature closure / Conclude | Sufficient understanding exists; offer closure without forcing it. |
| User overloaded + confused | Pause + Acknowledge | Reduce pressure; integrate what was gained before asking for more. |
| Spiralling + broad conversation | Progressively narrow focus | Investigation is widening without depth; redirect to high-value direction. |
| Understanding static + steady progress | Decide direction | Facts accumulate but insight does not — the line of enquiry needs repositioning. |
| Strong contradiction across domains | Challenge assumption + Clarify | Multiple independent lines of enquiry converge on a tension that should be surfaced. |
| Rapid evidence accumulation | Observe pattern | Multiple new items in quick succession suggest an emerging structure to note. |

---

## Relationship to Existing Layers

| Layer | Answers | Feeds Into |
|-------|---------|------------|
| Reasoning Graph | What does the system know and how does it know it? | Investigation Narrative |
| Investigation Narrative | What is known in human language? | Investigation State Assessment |
| **Investigation State Assessment** | What kind of help is appropriate right now? | Behaviour Selection |
| Facilitator Behaviour | What should we do next? | Facilitated Conversation |

The assessment translates *state* (narrative) into *readiness* (behaviour selection). It does not translate into action — that is the next layer's job.

---

## Emergent Architecture

The Confidence Engine architecture is becoming:

```
User
  ↓↑
Facilitated Conversation (where behaviour lives)
  ↓↑
Behaviour Selection (consumes assessment output)
  ↓↑
Investigation State Assessment (describes investigation)
  ↓↑
Investigation Narrative (human representation of state)
  ↓↑
Reasoning Graph (machine representation)
  ↓↑
LLM / Ollama / Reasoning Engine
  ↓↑
User
```

Each arrow is a data flow. Each layer has a single responsibility. The assessment layer does not decide, reason, present, or converse — it describes the investigation's current state through multiple analytical dimensions so that behaviour selection can act on state rather than implementation details.

This architectural description was captured during Experiment 18 and preserved as evidence of the assessment capability's design rationale. The capability is implemented in code but remains passive/experimental — it does not control the active user-facing investigation path. Future experiments may integrate these signals or replace them; this document records the design that was tested.