confidence-engine/docs/investigation-state-assessment.md

Investigation State Assessment — Architectural Specification

This is a design document only. Do not implement yet.

---

Purpose

The facilitator behaviour model (Experiment 15) defines what an expert consultant does during an investigation. It does not define how the system decides which behaviour to deploy.

This document introduces an architectural layer that sits between the Investigation Narrative and Behaviour Selection: the Investigation State Assessment.

Its purpose is to answer one question before any behaviour is selected:

Given where the investigation is right now, what kind of help is most appropriate?

The assessment does not decide. It describes. The decision process consumes its output.

---

Architecture

User
  ↓↑
Facilitated Conversation (behaviour deployed here)
  ↓↑
Behaviour Selection (consumes assessment output)
  ↓↑
Investigation State Assessment (evaluates investigation)
  ↓↑
Investigation Narrative (translates graph to human state)
  ↓↑
Reasoning Graph (machine representation)
  ↓↑
LLM / Ollama / Reasoning Engine
  ↓↑
User

The assessment layer sits between narrative and behaviour selection.

It reads investigation state from the narrative. It produces a structured evaluation that behaviour selection consumes.

It does not inspect graph nodes directly.

---

The Assessment Problem

Without an assessment layer, behaviour selection has two flawed options:

1. Inspect the graph directly. This couples behaviour to reasoning implementation. Any graph schema change breaks behaviour decisions.
2. Use narrative fields as proxy signals. This is fragile because narrative was designed for presentation, not decision-making.

The assessment layer exists so that behaviour selection depends on investigation state rather than implementation details.

---

Assessment Dimensions

Each dimension describes what the facilitator should know before acting.

For each: purpose, observable signals, possible values, and how behaviours may consume it.

---

1. Current Investigation Phase

Purpose: Determine which phase of investigation is active. Phase constrains which behaviours are appropriate.

Observable signals:

• How much evidence has been gathered relative to the situation's complexity
• Whether known items are mostly broad or detailed
• Whether active unknowns target foundational or peripheral questions
• The spread and depth of resolved subgraphs in the narrative

Possible values:

• Orienting — initial understanding being built, nothing established yet
• Exploring — breadth phase, multiple directions open
• Focusing — patterns emerging, directions narrowing
• Deepening — specific areas under scrutiny
• Synthesising — understanding coalescing, connections forming
• Concluding — sufficient resolution for current scope

How behaviours consume it:

• Orient is only appropriate in Orienting phase
• Exploring favours breadth behaviours: Clarify, Observe pattern, Connect
• Focusing favours: Decide direction, Progressively narrow focus
• Deepening favours: Challenge assumption, Validate, Refine understanding
• Synthesising favours: Summarise, Observe pattern, Communicate confidence honestly
• Concluding favours: Avoid premature closure, Communicate confidence honestly

---

2. Investigation Progress

Purpose: Determine whether the investigation is moving forward and at what velocity. Progress state affects pacing and behaviour urgency.

Observable signals:

• Rate of new resolved items per turn
• Whether recent turns add fundamentally new information or refine existing understanding
• Whether active unknowns are being resolved or replaced by new ones
• Depth of reasoning paths from conclusions to established facts

Possible values:

• Accelerating — each turn adds meaningful new resolution
• Steady — consistent, predictable progress
• Stalled — turns do not advance understanding
• Looping — similar questions or responses repeat across turns
• Spiralling — investigation widens without deepening in any direction

How behaviours consume it:

• Accelerating may trigger: Refine understanding, Observe pattern
• Steady is stable — continue current behaviour track
• Stalled triggers intervention: Challenge assumption, Decide direction, Pause
• Looping triggers: Pause, Clarify, or reorient the line of enquiry
• Spiralling triggers: Progressively narrow focus, Decide direction

---

3. Evidence Quality

Purpose: Determine the reliability and coherence of what has been established. Evidence quality determines whether the facilitator should push for more or consolidate.

Observable signals:

• Source diversity of evidence (single assertion vs. multiple converging sources)
• Presence of contradictions between established items
• Whether evidence relies on untested assumptions as premises
• Strength of reasoning paths connecting findings to conclusions

Possible values:

• Weak — little reliable evidence, mostly assertions or single-source information
• Mixed — some strong evidence alongside weaker or contested items
• Strong — multiple converging sources, coherent narrative, tested assumptions
• Contradictory — established items conflict in ways that have not been resolved

How behaviours consume it:

• Weak triggers: Clarify, Validate (what little exists), Challenge assumption
• Mixed triggers: Validate (strong items), Challenge assumption (weak items)
• Strong favours: Connect, Observe pattern, Refine understanding, Communicate confidence honestly
• Contradictory triggers: Challenge assumption, Expose uncertainty, Clarify

---

4. Understanding Trajectory

Purpose: Determine whether the investigation's understanding is growing, static, or degrading. This affects whether to push forward or consolidate.

Observable signals:

• Number of new coherent insights per turn (not just facts, but meaningful connections)
• Whether new information clarifies previous items or introduces new confusion
• Whether active unknowns are becoming better defined or more vague over turns
• The ratio of synthesized understanding to raw evidence accumulation

Possible values:

• Growing — each turn produces both facts and meaningful synthesis
• Static — information accumulates without deeper understanding forming
• Confused — new inputs introduce ambiguity rather than clarity
• Consolidating — understanding is stabilizing, fewer net insights but more coherence

How behaviours consume it:

• Growing favours: Connect, Observe pattern, Progressively narrow focus
• Static triggers: Challenge assumption, Decide direction, Pause (ask if user needs prompting)
• Confused triggers: Clarify, Expose uncertainty, Pause (hold space for clarity)
• Consolidating favours: Refine understanding, Summarise, Communicate confidence honestly

---

5. Uncertainty Trend

Purpose: Determine whether the investigation's overall uncertainty is increasing, reducing, or stable. This affects pacing and whether to push toward resolution.

Observable signals:

• Resolution ratio trending up vs. down across recent turns
• Whether new unknowns are being introduced faster than old ones resolved
• The severity (not just count) of remaining active unknowns
• Proportion of critical-path items still unresolved vs. peripheral items

Possible values:

• Increasing — more or deeper uncertainties emerging than resolving
• Reducing — clear trajectory toward resolution
• Stable — uncertainty holding steady, neither improving nor worsening
• Asymmetric — some areas well-resolved while others remain deeply uncertain

How behaviours consume it:

• Increasing triggers: Decide direction (redirect), Challenge assumption, Pause
• Reducing favours: Validate, Summarise, Communicate confidence honestly
• Stable is neutral — continue current track, monitor for change
• Asymmetric triggers: Expose uncertainty, Decide direction

---

6. Conversation Health

Purpose: Evaluate the quality of the interaction pattern between user and facilitator. Poor conversation health indicates the current approach needs adjustment regardless of investigation state.

Observable signals:

• Repetition of question types or response patterns across turns
• User response length trending shorter (disengagement) or longer (confusion, over-explaining)
• Whether the user is answering what is asked or redirecting to different topics
• Frequency of meta-comments ("What are we trying to find out?")

Possible values:

• Healthy — natural back-and-forth, appropriate depth, engaged responses
• Repetitive — similar question-response patterns repeating without progress
• Too broad — user responses cover too much ground, losing focus
• Too narrow — investigation compressed to one dimension, missing context
• User overloaded — user asked to process too much per turn
• User under-informed — user lacks sufficient context to give useful answers

How behaviours consume it:

• Healthy is stable — continue current track
• Repetitive triggers: Pause, Clarify, Decide direction (change approach)
• Too broad triggers: Progressively narrow focus, Decide direction
• Too narrow triggers: Expose uncertainty, Challenge assumption, Connect
• User overloaded triggers: Pause, Acknowledge (reduce pressure), Communicate confidence honestly
• User under-informed triggers: Clarify, Orient, Expose uncertainty

---

7. Behaviour Readiness

Purpose: A derived signal — not assessed independently but synthesized from the dimensions above. It answers: Which behaviours are available and appropriate right now?

This is the assessment's output layer. Each dimension feeds into behaviour readiness as a set of weighted signals rather than a single verdict.

Signals produced (not decided):

• Which behaviours are available (phase permits)
• Which behaviours are pressed for (multiple dimensions converge on the same intervention)
• Which behaviours are inappropriate (contraindicated by current state)
• Which behaviours are stable (appropriate across all or most states)

How behaviours consume it: Behaviour selection does not read individual dimensions. It reads behaviour readiness as a single structured signal and selects accordingly. This keeps behaviour independent from assessment implementation details.

The facilitator should understand the readiness layer as a lens — an interpretation of raw state through multiple analytical dimensions — rather than as a decision mechanism itself. The readiness signals inform; they do not decide.

---

Assessment Principles

Principle 1: Assess, Do Not Decide

The assessment describes the investigation. It does not select behaviours.

Purpose: Separate evaluation from action. A cleaner separation means each layer can evolve independently.

Observable signals: N/A — this is an operating constraint on the layer itself.

How behaviours may consume it: Behaviours read assessment output as state, not as instructions. The same assessment should support multiple behaviour selection strategies (deterministic rules, weighted scoring, or LLM-assisted). The assessment does not lock in any particular strategy.

---

Principle 2: All Signals Traceable to Narrative

Every assessment signal must be derivable from the Investigation Narrative without reading the graph directly.

Purpose: Keep behaviour decoupled from reasoning implementation. If the graph schema changes, the assessment does not need to change — as long as the narrative preserves its fields.

Observable signals: The narrative has all necessary data: phase indicators, resolution counts, evidence quality signals, turn history, response patterns. None of these require graph traversal.

How behaviours may consume it: Behaviours trust the assessment as a stable contract. If the assessment accurately reflects investigation state through narrative fields, the behaviour layer remains correct regardless of how the narrative is derived.

---

Principle 3: Signals Are Descriptive, Not Prescriptive

Each signal describes what is happening, not what should be done.

Purpose: Prevent the assessment from becoming a decision tree in disguise. A descriptive assessment supports multiple interpretation strategies; a prescriptive one locks into one.

Observable signals: Each dimension's "possible values" are neutral descriptions. "Stalled" does not mean "use Pause" — it means "progress has stopped." What to do about that is a behaviour selection question.

How behaviours may consume it: Different investigation contexts may interpret the same signal differently. Stalled in early phase might mean "need more context"; in late phase it might mean "reach closure." The assessment does not encode either interpretation.

---

Principle 4: Convergence Matters More Than Any Single Signal

A behaviour should be selected when multiple dimensions converge, not when one dimension reaches a threshold.

Purpose: Prevent brittle, single-signal triggers that produce inappropriate responses in edge cases. Convergent signals indicate robust state patterns.

Observable signals: Cross-dimensional correlation — e.g., Looping progress + Repetitive conversation + Confused understanding is a stronger signal than any one alone.

How behaviours may consume it: The behaviour readiness layer should express convergence explicitly: which dimensions agree on which intervention area, and to what degree. Behaviours are more confident when multiple signals align.

---

Principle 5: Assessment Is Stateful Across Turns

The assessment is not recomputed from scratch each turn. It accumulates and evolves.

Purpose: Investigation state is temporal. Current status alone cannot capture looping, acceleration, or convergence. Turn history is essential to meaningful assessment.

Observable signals: Turn-level deltas (what changed), sequence patterns (what repeated), trend direction (accelerating, decelerating), phase transitions.

How behaviours may consume it: Behaviours that depend on temporal patterns (Looping detection, Progressive narrowing) need the assessment to carry forward state between turns. A turn-by-turn stateless assessment cannot detect these patterns.

---

Principle 6: Uncertainty About Assessment Is Itself Assessable

The assessment should be able to express its own uncertainty about dimensions it cannot reliably evaluate.

Purpose: Avoid false precision. Some investigation states are genuinely hard to classify (e.g., early phase with ambiguous inputs). The assessment should say "I cannot determine this reliably" rather than guessing.

Observable signals: Insufficient data, conflicting signals across dimensions, rapid state changes within a single turn, user input that does not map cleanly to existing categories.

How behaviours may consume it: Behaviours should handle uncertain assessment fields gracefully — defaulting to conservative, non-committing actions (Acknowledge, Clarify) when the assessment cannot determine direction.

---

Decision Matrix

An exploratory design aid. Shows which investigation states most naturally lead to which behaviour types. These are patterns, not rules.

Investigation State	Likely Behaviour	Reason
Orienting + stalled	Clarify	Too many unknowns remain; the starting point needs anchoring.
Exploring + broad	Observe pattern	Multiple data points suggest a theme before pushing for specifics.
Focusing + asymmetric uncertainty	Expose uncertainty	Knowing some areas well while others remain fuzzy demands visibility.
Deepening + strong evidence	Validate	Solid findings should be marked and integrated before adding more.
Deepening + contradictory evidence	Challenge assumption	Evidence conflicts; the foundation of one or more lines of enquiry is shaky.
Synthesising + reducing uncertainty	Refine understanding / Summarise	Shared understanding is coalescing; compress without losing detail.
Any phase + looping detected	Pause	Risk of conversational repetition; hold space for reflection.
Convergent high evidence quality	Communicate confidence honestly	The state warrants explicit confidence language — match epistemic reality.
Low uncertainty + sufficient resolution	Avoid premature closure / Conclude	Sufficient understanding exists; offer closure without forcing it.
User overloaded + confused	Pause + Acknowledge	Reduce pressure; integrate what was gained before asking for more.
Spiralling + broad conversation	Progressively narrow focus	Investigation is widening without depth; redirect to high-value direction.
Understanding static + steady progress	Decide direction	Facts accumulate but insight does not — the line of enquiry needs repositioning.
Strong contradiction across domains	Challenge assumption + Clarify	Multiple independent lines of enquiry converge on a tension that should be surfaced.
Rapid evidence accumulation	Observe pattern	Multiple new items in quick succession suggest an emerging structure to note.

---

Relationship to Existing Layers

Layer	Answers	Feeds Into
Reasoning Graph	What does the system know and how does it know it?	Investigation Narrative
Investigation Narrative	What is known in human language?	Investigation State Assessment
Investigation State Assessment	What kind of help is appropriate right now?	Behaviour Selection
Facilitator Behaviour	What should we do next?	Facilitated Conversation

The assessment translates state (narrative) into readiness (behaviour selection). It does not translate into action — that is the next layer's job.

---

Emergent Architecture

The Confidence Engine architecture is becoming:

User
  ↓↑
Facilitated Conversation (where behaviour lives)
  ↓↑
Behaviour Selection (consumes assessment output)
  ↓↑
Investigation State Assessment (describes investigation)
  ↓↑
Investigation Narrative (human representation of state)
  ↓↑
Reasoning Graph (machine representation)
  ↓↑
LLM / Ollama / Reasoning Engine
  ↓↑
User

Each arrow is a data flow. Each layer has a single responsibility. The assessment layer does not decide, reason, present, or converse — it describes the investigation's current state through multiple analytical dimensions so that behaviour selection can act on state rather than implementation details.

This is an emerging architectural direction. It is intentionally recorded before implementation so future experiments remain aligned.